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TRAINING  RESEARCH  PROGRAM  AND  PLANS; 

ADVANCED  SIMULATION  IN  UNDERGRADUATE  PILOT  TRAINING  (ASUPT) 


1.0  INTRODUCTION  AND  BACKGROUND 

The  Homan  Resoir  .es  Laboratory,  Flying  Training  Division  (HRL/FT) 
has  developed  a unique  capability  for  conducting  research  on  Important 
methods  and  equipment  problems  In  pilot  training.  Although  the  present 
orientation  of  the  laboratory  Is  toward  the  solution  of  undergraduate 
pilot  training  problems  many  of  these  study  results  will  be  general izable 
to  other  training  programs,.  Further,  the  flexibility  of  the  HRL/FT 
facility  makes  It  possIbU  to  carry  out  Investigations  of  many  problems 
of  training  methodology  and  equipment  unique  to  other  pilot  training 
programs. 

In  planning  the  HRL/FT  facility,  contract  studies  were  conducted 
to  specify  the  training  problems  to  be  Investigated  and  the  organization 
and  equipment  requirements  for  conducting  these  investigations.  The 
contract  results  were  reviewed  and  revised  by  Air  Force  personnel  and 
formulated  into  a basic  working  document  entitled  "ASUPT  Utilization 
Concept".  Under  this  concept  Investigations  were  centered  primarily  up- 
on the  training  problems  and  training  device  characteristics  necessary 
for  an  effective  undergraduate  pilot  training  program  to  be  carried  out 
by  HRL/FT.  Consideration  was  also  given,  although  not  so  expllclty.,  to 
the  training  problems  peculiar  to  programs  beyond  undergraduate  pilot 
training. 

As  Indicated,  Human  Resources  Laboratory  research  management  planning 

focused  first  upon  research  designed  to  Increase  the  effectiveness  of 
the  Undergraduate  Pilot  Training  program  ItlPT).  However,  management 
was  equally  concerned  that,  to  the  maximum  extent  possible,  the  research 
be  carried  out  In  a way  which  would  make  the  results  general Izable  to 
other  levels  of  pilot  training.  These  considerations  led  to  the  de- 
cision to  Initiate  a contractor  conducted  study  to  Identify  these 
general  training  research  questions.  These  questions  were  Identified 
through  conduct  of  a survey  of  those  Individuals  In  the  training 
community  selected  for  their  experience  with  the  field.  This  panel  of 
experts  was  composed  of  those  Individuals  In  government  and  Industry 
who  were  recognized  as  experienced  and  knowledgeable  and  who  could  de- 
vote the  necessary  time  to  the  program.  The  panel  members  were  asked 
to  Identify  those  research  questions  which  they  felt  to  be  pressing  In 
terms  of  their  Impact  upon  pilot  training  effectiveness.  They  were 
asked,  also,  to  order  these  questions  in  terms  of  their  Importance  to 
increasing  the  effectiveness  of  beginning  pilot  training’ programs. 

The  study  reported  here  is  the  result  of  work  toward  the  goal  of 
defining  those  priority  resear-.h  investigations  to  be  undertaken 
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using  the  research  capabilities  of  HRL/FT.  Specifically  the  study  en- 
compassed the  following  major  elements: 

1.  A statement  of  the  major  training  research  questions. 

2.  An  assignment  of  priorities  by  knowledgeable  individuals 
to. these  resezirch  questions  In  terms  of  their  Importance 
to  increasing  training  effectiveness* 

3.  Examination  cf  the  research  questions  In  light  of  the 
research  capabilities  of  HRL/FT, 

4.  Restructuring  the  priorities  based  upon  the  research 
capabilities  and  constraints. 

rJ,  Setting  forth  the  details  of  those  experiments  and 
investigations  to  be  undertaken  most  immediately. 


2.0  METHOD 

The  method  used  in  carrying  out  the  work  may  be  described  under 
three  major  categories  of  effort.  These  were  (a)  development  of  a 
priority  list  of  research  questions  In  term;;  of  their  Impact  upon  train- 
ing effectiveness,  (b)  examination  of  the  capabilities  of  the  research 
facility  and,  (c)  delineation  of  the  high  priority  experiments,  and  In- 
vestigations to  be  carried  out  using  the  facility. 

2. I Development  of  a Priority  List  of  Research  Questions 

The  development  of  the  list  of  research  questions  and  the  deter- 
mination of  the  priorities  of  the  Items  on  the  list  was  carried  out  In 
two  phases.  During  the  first  phase  a list  of  research  questions  was 
generated  through  review  of  the  relevant  current  literature,  and  most 
Importantly,  by  a panel  of  selected  Individuals.  These  Individuals  were 
generally  recognized  as  experts  In  the  field  of  pilot  training  and 
research.  Their  background  reflected  experience  In  government  agencies, 
ur (versltles  and  private  Industry.  These  panel  members  were  contacted 
In  person  or  by  telephone  and  a follow-up  letter  explaining  the  project 
In  detail,  was  sent.  A copy  of  this  letter  is  given  In  Appendix  A. 

Twelve  panel  members  submitted  research  questions  which  they  believed 
to  be  most  relevant  to  be  Investigated  in  order  to  Increase  the  effective- 
ness of  pilot  training.  In  all,  a list  of  fifty-five  research  questions 
was  generated.  These  research  questions  were  not  equal  with  respect  to 
the  breadth  or  scope  of  effort  which  would  be  required.  They  were,  how- 
ever, those  questions  which  were  deemed  important  enough  by  this  group  of 
experienced  individuals  to  be  Included  in  a research  program.  These 
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questions  were  defined  and  explained  as  concisely  and  clearly  as  possible 
and  re-submftted  as  a total  list  to  each  of  the  individuals  on  the  panel 
for  their  Judgments  as  to  the  priority  rankings  of  the  questions.  The 
definitions  of  the  research  items  are  given  in  Appendix  B,  The  Instruc- 
tions to  the  raters  tfor  assigning  priorities  are  given  In  Appendix  C. 

The  method  suggested  to  the  panel  members  for  assigning  priorities 
was  a modified  paired  comparison  technique.  This  technique  required 
that  each  item  be  listed  on  a 3x5  card.  The  rater  selected  one  item  and 
then  sorted  the  remaining  items  as  to  whether  each  Item  was  of  greater 
priority  or  lesser  priority  than  the  Item  selected.  This  procedure  re- 
sulted In  two  groups  of  items;  one  of  higher  priority  and  another  of  lower 
priority  than  the  selected  item.  Each  of  these  groups  were  then  sorted 
in  a similar  manner  and  the  resulting  groups  further  sorted  In  the  same 
way  until  groups  of  four  or  five  items  were  attained.  These  items  were 
then  ranked  as  a group  and,  finally,  all  items  assembled  into  a priority 
list.  Ten  panel  members  assigned  priorities,  two  being  unable  to  because 
of  other  commitments.  The  results  of  these  listings  and  the  priority 
ratings  are  given  and  discussed  in  Section  3.0. 

2.2  Examination  of  the  Ca  .abilities  of  the  Research  Facility 

The  objective  of  this  phase  of  the  contract  effort  was  to  examine 
the  AFHRL/FT  research  capabilities  and  constraints  so  that  particular 
research  investigations  could  be  detailed  and  recommended.  These  recom- 
mendations could  then  be  based  on  these  capabilities  and  constraints,  as 
well  as  the  results  from  the  panel  of  experts. 

The  items  of  research  equipment  at  the  AFHRL/FT  facility  will  be 
identified  here  only  in  general  terms  since  they  are  described  in  detail 
in  Hagln  and  Smith  (1974). 

The  major  Item  of  research  equipment  is  the  Advanced  Simulation  in 
Undergraduate  Pilot  Training  (ASUPT)  device.  ASUPT  has  come  to  mean  the 
large  scale  two  cockpit  full  motion  capability  simulator  with  a 7 channel 
computer  generated  visual  system.  This  device  is  described  on  pages  17 
through  26  in  Hagln  and  Smith,  To  summa.  ize.  It  has  a six  degree  of 
freedom  motion  system  and  an  external  world  visual  system  with  a field 
of  view  which  essentially  duplicates  that  of  the  T-37  aircraft  which  it 
represents.  The  Computer  Image  Generator  (CIG)  produces  a vsry  extensive 
variety  of  objects  for  making  up  the  visual  scene  for  the  pilot. 

The  ASUPT  device  incorporates  a number  of  advanced  Instructional 
features  which  may  be  tested  for  their  contribution  to  teaching  effective- 
ness. These  capabilities  include  selective  task  sequencing;  variable 
task  difficulty  and  complexity;  selective  malfunction  insertion; 
simulator  freeze;  rapid  ro-initlalization;  automated  demonstration;  seif- 
confrontation;  and  a number  of  methods  for  providing  knowledge  of  results 
to  the  trainee. 


3 


it  a 


The  total  HRL/FT  training  research  facility  Includes  several  de- 
vices less  complex  than  the  ASUPT  which  makes  it  possible  to  Investigate 
certain  research  questions  using  these  less  complicated  and  expensive 
devices.  The  first  of  these  Is  the  T-4G  trainer.  This  device  Is  a two 
degree  of  freedom  motion  platform  trainer  with  a Singer  SPO  Electronic 
Perspective  Transformation  (EPT)  visual  system.  The  Image  generation 
system,  based  on  a color  movie  film,  uses  a CRT  to  project  an  electron- 
ically generated  horizon  display  such  that  an  approach,  iandlng  and  take- 
off soquence  may  be  presented  as  a visual  display  for  the  trainee. 

Another  Important  Item  of  research  equipment  Is  the  T-40  trainer. 
This  trainer  Is  an  Instrument  and  procedures  trainer  incorporating  a two 
degree  of  freedom  motion  system.  It  Is  a slde-bv-slde  seating  cockpit 
with  an  Instrument  configuration  similar  to  that  Installed  In  the  T-39 
aircraft. 


A further  research  capability  In  the  HRL/FT  facility  Is  the  Formation 
Flight  Trainer  (FFT),  This  trainer  Is  a fixed  base  low  fidelity  part- 
task  trainer  which  provides  some  of  the  essential  visual  cues  for  teaching 
undergraduate  pilots  the  basic  formation  flight  skills.  The  cockpit  has 
a stick,  throttle,  rudder  pedals  and  simple  instrument  displays  and  the 
trainer  Incorporates  the  rudimentary  flight  dynamics  of  the  T-38  aircraft. 
A spherical  screen  Is  used  to  provide  a 200  degree  horizontal  and  90  de- 
gree vertical  field  of  view  upon  which  Is  projected  on  Image  of  either  a 
T-37  or  T-38  model  aircraft.  The  trainee  is  given  the  opportunity  to 
practice  flying  formation  from  observation  of  the  projected  Image  of  this 
aircraft  on  the  screen. 


An  Important  capability  of  the  HRL/FT  research  facility  Is  the  Auto- 
matic Pata  Acquisition  and  Control  System  (ADACS),  used  In  connection  with 
the  T-4G,  T-40  and  FFT  trainers.  This  Is  a system  which  has  as  Its  basic 
unit  a System  Engineering  Laboratory  (SEL)  72  computer.  The  ADACS  Is 
capable  of  sampling  data  from  the  T-40  trainer,  the  T-4G  and  the  FFT  and 
processing  It  for  up  to  29  measurement  parameters.  The  ADACS  permits 
accomplishment  of  research  in  the  area  of  recording  techniques,  perform- 
ance measures  and  proficiency  measurement  using  both  on-line  and  off-line 
techniques.  The  ADACS  facility  can  bo  Interfaced  with  the  SEL  86  com- 
puters which  are  the  core  computers  for  the  ASUPT  device.  Together  the 
SEL  86  computers  and  the  SEL  72  comprise  a powerful  computational  cap^ 
ability  for  the  HRL/FT  facility. 

Ofher  capabilities  within  the  HRL/FT  facility  support  research  both 
In  the  determination  of  training  device  requirements  and  in  training 
methods  and  training  device  utilization.  The  first  of  these  is  un  eye 
movemenl  recording  capability  which  has  been  developed  by  HRL/FT  personnel. 
The  device  can  .e  used  both  In  aircraft  and  simulators  and  is  designed 
to  provide  data  with  respect  to  the  information  or  impor+ant  content 
of  the  display  used  by  the  pilot  and  the  required  field  of  view.  A 
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second  capability  Is  the  laboratory’s  audio-visual  Instructional  media 
facilities,  "'dse  are  comprised  of  a video  laboratory  for  the  pro- 
duction of  video  Inst. uct Iona!  presentations,  a series  of  learning  center 
carrels  and  the  Audlo-V Isral  Instrument  Training  device  (AVIT)  developed 
by  Life  Sciences,  l'*r  Inis  latter  device  has  the  capability  for  pre- 
senting both  visual  end  ,iudlo  Information,  forward  and  backward  branch- 
ing In  both  audio  an'  ? *s ! and  Interaction  with  th*  device  by  the 

stvdMnl  bo^h  with  afresv'^  controls  and  multiple  choice  answer  response 
kay.y. 


A ■final  major  arid  . 'nort&nv  fl^re  of  equipment  available  to  the 
iiRL/FT  facility  Is  an  !»,. 1 'up..ircd  \ *37  aircraft.  This  aircraft  Is  a 
vary  Important  piece  of  equipae*?  for  obtaining  basic  Information 
relevant  to  the  pllov‘s  overall  system  performance  and  Is  essential  to 
Vho  t j«p,t  of  the  per-.,  ■ -sneo  eqwtvM wee  thesis  discussed  In  Section  4.0. 
Mbs'1,  lmu''it».ntly,  '’'ows  for  the  ~o~.'*lation  of  the  level  of  the 
pilot’,  performance  In  cont-ol  s sng  sys  it  parameters  such  as 

bcv'/dl* and  al.*sp*.'<1  v’th  antrol  i-.ou*  pararrmrers  such  as  fore-and-aft 
’ ‘-oral  stick  mov^rtents.  T;*yse  measures  are  essential  to  establish- 
ing i base  fo>  tfr>  . f.Yarmlnat-ori  of  the  equivalence  of  ths  ASUPT 
device  to  the  tire  "ft  i>0  simulated. 

2,3  foundation  of  the  H .-ft  Priority  Experiments 

Based  upon  the  priority  assignment  by  the  panel  of  experienced 
individuals  and  +he  capabilities  of  the  research  equipment  and  constraints 
surrounding  the  research  facility,  certain  experiments  and  supporting 
Investigations  ha\»  besr>  delineated.  In  the  delineation  of  these  ex- 
perlraints  It  was  felt  that  the  highest  effectiveness  possible  should  be 
made  In  the  use  of  the  research  equipment  and  personnel.  Therefore,  a 
second  group  of  Individuals  with  experience  in  experimental  laboratory 
and  f!  Id  Investigations  was  selected  to  provide  expertise  In  how  thess 
experiments  should  be  conducted.  It  was  a function  of  this  panel  to 
previde  guidance  in  designing  these  experiments  so  that  they  would  be 
rn^at  economical  and  most  effective  In  attaining  results  which  could  be 
used  wlln  confidence  in  decisions  with  respect  to  training  proct^res 
and  training  methods.  These  experiments  are  dellnoated  In  a later  section. 

It  was  also  felt  necessary  to  examine  the  research  questions, 
particularly  those  having  to  do  with  training  methodologies  and  progression 
of  the  trainee  through  the  training  program,  utilizing  less  complex  and 
expends-!  deices  to  the  maximum  extent  possible.  It  was  felt  that  much 
lr  >rsY(3t?on  directly  useful  for  Improving  the  effectiveness  of  the  Undei - 
aduate  Dllot  Training  program  could  be  gained  through  direct  use  and 
experimentation  with  the  simpler  devices.  For  example,  earlier  work  with 
the  use  of  the  T-4G  simulator  In  the  T-37  Undergraduate  Pilot  Training 
program  has  demonstrated  the  worth  of  such  devices  In  determining  the 
contribution  of  different  training  approaches,  (see  Woodruff,  Smith  & 
Morris,  1972).  They  have  also  proved  valuable  In  The  Investigation  of 
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system  performance  measures  and  are  recommended  to  be  put  to  further  use 
In  the  Investigation  of  operator  output  performance  measurement.  The 
audio-visual  capability  of  HRl/FT  along  with  devices  such  as  the  AVIT 
are  important  to  Investigations  of  training  In  specific  training  objec- 
tives which  form  the  basis  for  accomplishing  the  final  training  objective 
for  the  undergraduate  pilot. 

3.0  PRIORITY  RATfNGS  OF  RESEARCH  PROBLEMS 

The  priorities  assigned  by  the  panel  of  experienced  training  research 
individuals  were  used  to  generate  an  initial  listing  of  problems  Judged 
to  be  important  to  increasing  the  effectiveness  of  pilot  training.  This 
list,  taken  In  conjunction  with  the  capabilities  of  the  HRL/FT  facility, 
provided  +he  basis  for  deriving  a list  of  recommended  Investigations, 

As  was  described  earlier,  the  members  of  the  panel  were  asked  to 
arrive  at  a listing  of  the  Items  using  a modified  paired  comparison 
technique  such  that  high  priority  It, ms  could  be  Identified,  This 
technique  Is  described  In  dotal  I In  the  Instructions  to  the  raters  given 
In  Appendix  C. 

The  median  ratings  for  the  Items  were  computed  and  are  tabled  In 
Appendix  D,  along  with  their  semi -Interquart l ie  range  (Q)  values.  An 
abbreviated  list  In  Judged  order  of  Importance  Is  given  In  Table  3.1. 

This  list  Is  derived  f r jm  the  panel  list  and  considerations  of  admlnls- 
trarive,  scheduling  and  equipment  constraints.  It  will  be  discussed  In 
detail  in  later  sections. 


Table  3. i 

Abbreviated  List  of  First  Priority  Research  vuestlons 
Formulated  on  Basis  of  Multiple  Criteria 


1,  Motion-Vision  Interaction 

2.  Content  of  the  Visual  Display 

3.  Sequencing  of  Training  Tasks 

4,  Cognitive  Pre-Training 

5U  Individualized  Instruction 

6.  Feedback 

7.  Instructor  Training 

8.  Contextual  Training 


In  considering  the  Items  In  these  lists  an  examination  of  the  seml- 
Interquartl le  range  values  listed  under  the  Q column  Is  In  order.  These 
values  reflect  the  variability  among  the  ratings  for  the  several  Items. 
They  show  considerable  variability  In  the  ratings  of  the  Items  given  by 
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the  panel  Members.  To  a great  extent,  thts  variability  arises  from 
genuine  differences  of  opinion  among  the  raters  as  to  which  problems  are 
more  Important  for  solution  In  Improving  the  effectiveness  of  ptlot 
training.  However,  some  of  the  variability  comes  from  the  seemingly  in- 
escapable intrusion  of  other  criterion  Into  the  rating  procedure.  Since 
the  panel  members  have  been  working  first  hand  with  on-going  training 
programs  and  carrying  out  training  research  In  operational  settings, 
they  found  it  difficult  in  some  cases  to  assign  priorities  on  a purely 
detached  "contribution  to  training  effectiveness"  basis.  For  example, 
some  judgss  felt  that  certain  items  required  more  effort  in  pulling  to- 
gether relevant  information  in  the  literature  and  in  applying  that 
information  to  the  problem.  The  feeling  was  that  if  this  were  done, 
the  Item  would  not  command  a high  priority  in  an  on-going  research 
program.  Some  judges  felt  that  certain  items  wore  so  Important  on  other 
bases,  such  as  pfiot  acceptance,  that  tt  would  not  be  worthwhile  to  ex- 
pand effort  on  evaluating  them  In  terms  of  contributions  to  training  per 
so.  Again,  some  Judges  felt  that  for  certain  items  such  as  aircraft 
dynamics  simulation,  the  engineering  developments  were  such  that  research 
as  to  Its  relationship  to  training  effectiveness  could  not  be  considered 
high  priority.  That  is  to  say,  that  providing  high  fidelity  in  the  air- 
craft dynamics  simulation  was  fait  to  be  so  feasible  and  relatively 
economical  that  research  on  the  fidelity  of  aircraft  dynamics  simulation 
was  not  particular^  urgent. 

Thj  comrwn+s  and  annotations  by  the  panel  of  judges  In  assigning 
the  priorities  to  the  research  Items  are  Important  In  Interpreting  the 
priority  list.  However,  those  Items  which  appear  In  the  top  ten  of  the 
list  In  Appendix  D are  not  dissonant  with  the  general  conclusions  to  be 
drawn  from  discussions  with  the  Individual  Judges.  This  Is  also  true  of 
the  icw  priority  items  In  the  list. 

It  was  Indicated  In  Section  1.0  that  the  research  Investigations  to 
bo  undertaken  as  first  priority  would  be  selected  on  the  basis  of  the 
research  priorities  assigned  by  the  training  problem  and  other  consid- 
erations. These  additional  considerations  are:  the  capabilities  of  the 
research  facility,  the  criticality  of  a question  In  terms  of  timely  In- 
put Into  the  training  change  process,  and  administrative  considerations 
such  as  subject  and  research  personnel  availability  and  practicality  In 
scheduling.  These  considerations,  taken  along  with  the  experts'  ratings,, 
form  the  basis  for  an  overall  plan  for  undertaking  Investigations  and 
the  recommending  of  certain  first  priority  experiments.-  The  plan  and 
the  experiments  are  described  In  Soc+ion  5.0. 


T T'~  rA.--1l  l8l 


4.0  METHODOLOGICAL  CONSIDERATIONS 


4, 1 General  Considerations 

There  are  certain  considerations  which  It  Is  felt  should  be  dls- 
cussed  before  arriving  finally  at  a plan  for  long  range  training 
research.  The  first  of  thase  are  the  difficulties  inherent  in  carrying 
out  classical  transfer  of  training  experiments.  These  difficulties  make 
It  advisable  to  examine  alternate  methods  for  obtaining  experimental 
data  for  use  In  making  decisions  about  training  methods  and  training 
devices.  The  development  and  test  of  methods  which  may  provide  useful 
data  more  economically  and  In  more  timely  fashion  Is  an  area  which  Is 
recommended  as  worthy  of  Investigation  because  of  the  potential  savings 
in  money  and  time. 

The  difficulties  attendant  on  classical  transfer  studies  are  time, 
cost  and  availability  of  experimental  subject  trainees.  In  order  to 
assess  adequately  the  transfer  effects  of  any  given  variable  under  study, 
a representative  sample  of  a class  of  students  is  required  for  each  of 
the  experimental  groups  and  the  control  group.  The  calendar  time  re- 
quired for  an  experiment  can  vary  from  a short  period  when  the  effect  of 
training  In  the  simulator  upon  the  performance  of  a single  task  Is 
assessed  to  that  time  required  to  train  a class  In  a complete  curriculum 
as  would  be  the  case  when  Investigating  different  sequences  of  the  train- 
ing tasks.  When  the  experiments  I question  Is  that  of  the  configuration 
of  the  training  equipment  the  methodological  problems  are  somewhat  diff- 
erent from  those  In  which  the  research  addresses  training  methods* and 
equipment  utilization.  Let  us  first  oxamlne  the  training  methods  and 
equipment  utilization  research  area,  since  It  is  here  that  the  classical 
transfer  of  training  methods  are  most  applicable  and  will  need  to  be 
preserved. 

The  primary  problem  In  the  classical  transfer  approach  lies  In  the 
cost  and' calendar  time  required  for  transfer  to  the  aircraft  after 
training  In  the  simulator.  In  order  to  expedite  the  acquisition  of 
Information  relevant  to  the  effectiveness  of  different  training  methods 
and  ways  of  utilizing  the  simulator,  It  Is  proposed  that  studies  be 
undertaken  to  establish  the  full  ASUPT  simulator  as  a criterion  system 
to  be  used  In  those  transfer  of  training  studies  Investigating  training 
methodology  and  simulator  utilization. 

With  this  approach,  it  Is  essential  that  the  ASUPT  device  be 
established  as  a valid  representation  of  the  aircraft.  In  the  past  this 
has  been  done  In  two  major  complementary  ways.  The  first  of  these  Is 
to  Implement  the  mathematical  model  of  the  flight  characteristics  of  the 
aircraft  as, accurately  as  possible  In  the  simulator  computer.  The 
second,  Is  to  make  the  general  cue  environment  of  the  pilot  as  much  like 
that  of  the  aircraft  as  possible  using  the  criterion  of  asking  experl- 
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enced  pilots  to  fly  both  the  simulator  and  tha  aircraft  and  to  make 
judgments  as  to  their  similarity.  It  is  proposed  that  this  technique' 
may  be  strengthened  by  the  use  of  measurements  of  the  pi  lots’  control 
performance  while  flying  both  the  aircraft  and  the  simulator,  it  Is 
proposed  that  an  estimate  of  the  population  distribution  of  these  per- 
formance values  for  the  aircraft  be  established  through  collection  of 
data  in  an  instrumented  T-37  aircraft.  The  rationale  is  that,  If  the 
experienced  pilot  fties  the  aircraft  through  maneuvers  In  which  specified 
system  measures  are  required  to  be  wtthln  specified  limits  and  It  is 
shown  that  his  control  performance  In  the  ASUPT  while  performing  within 
identical  limits  is  the  same  as  in  the  aircraft,  this  is  supportive  data 
to  the  equivalence  of  the  simulator  to  the  aircraft  in  respects  important 
to  p5lot  oontr^!.  '>.s  c-vMine  of  a data  collection  program  in  the  air- 
craft is  dlsv... .*ea  ir:  S^rtlon  5.2,5. 

The  me  o r being  proposed  has  been  termed  the  performance  equiva- 
lence approach  to  establishing  the  characteristics  of  the  training 
simulator.  The  method  constitutes  a quantification  of  the  present 
methods  for  using  the  skids,  and  knowledges  of  highly  trained  and 
experienced  pilots  In  making  the  simulator  equivalent  to  the  aircraft 
in  terms  of  the  pilot  performance  required.  Establishing  the  utility 
and  vadddy  of  this  technique  In  the  ASUP'r  investigations  will  provide 
the  ground  work  for  Its  use  In  the  evaluation  and  calibration  of  other 
training  simulators  within  the  Air  Force. 

Performance  equivalence  in  terms  of  operator  performance  is  assumed 
to  be  established  upon  demonstration  that  his  performance  under  one  set 
of  conditions  is  not  different  from  his  performance  under  a reference 
set  of  conditions.  That  is  to  say  that  performance  measures  of  operator 
output  obtained  in  the  Instrumented  aircraft  may  be  used  as  an  estimate 
of  the  population  mean  lx)  and  the  standard  error  about  that  mean  (ox) 
for  specified  measures.  Confidence  limits  may  then  be  set  up  about  the 
mean  to  determine  whether  any  other  set  of  conditions.  I.e. . con- 
figurations of  the  simulator,  fad  within  or  withou4*  those  confidence 
limits  with  respect  to  the  performance  measures  being  taken.  !t  is 
necessary  to  ».int  out  that  the  performance  measures  in  question  must 
be  established  as  being  reliable,  discriminating,  and  logically  and 
meaningfully  related  to  the  task  at  hand. 

Once  the  ASUPT  system  has  been  established  as  being  representative 
of  the  criterion  aircraft  and  performance  equivalence  has  been  estab- 
lished, the  system  may  be  utilized  as  the  criterion  system  for  transfer 
of  training  experiments  in  which  the  aircraft  would  normally  be  used. 
Also,  it  provides  a criterion  system  for  research  in  situations  rhich 
are  not  practical  or  advisable  in  the  aircraft.  These  are  those  which 
either  pose  safety  hazards  or  impose  unusua>  stresses  on  the  aircraft. 

The  question  of  the  genera d zabi 1 ity  of  the  findings  regarding 
training  methods  obtained  in  this  manner  may  be  raised  at  this  point. 


1+  could  be  held  that  results  as  to  the  comparative  effectiveness  of 
different  training  methods  obtained  through  using  ASUPT  as  the  criterion 
system  «.rc  not  valid  for  training  for  irt  <sfer  to  the  ai rcraft.  if  this 
thesis  were  true  It  would  follow  that  the  'esuits  could  be  held  not  to 
be  valid  for  use  In  other  simulators  as  well  or  that  methods  used  in 
one  training  aircraft  are  not  useful  In  another.  Either  contention 
would  seem  to  be  contrary  to  the  body  of  evidence  and  practice  In 
education  and  training  in  which  training  methods  and  techniques  are 
valid  across  a variety  of  training  situations  and  training  materials 
and  are  not  unique  to  them. 

The  second  major  area  in  which  training  effectiveness  decisions 
must  be  made  Is  that  of  the  configuration  of  the  training  simulator. 

Here  again,  we  may  consider  the  use  cf  the  classical  transfer  of  train- 
ing paradigm  and  the  role  that  might  be  played  by  other  methodologies 
in  support  of  training  effectiveness  decisions.  When  carrying  out  a 
classical  transfer  of  training  study  in  which  the  question  is  being  asked 
as  to  the  relative  importance  to  transfer  of  training  of  given  levels  of 
the  characteristics  It  is  necessary  to  establish  levels  of  that  variable 
for-exper! mental  study.  For  example,  a transfer  of  training  study  might 
be  carried  out  In  which  all  combinations  of  degrees  of  freedom  of  move- 
ment of  the  simulator  are  selected  as  experimental  conditions.  The 
number  of  possible  experimental  conditions  and  trainees,  as  well  as  the 
amount  of  calendar  time  required  for  the  experiment  would  be  very  large 
(see  Smith,  1972).  Further,  and  importantly.  It  is  necessary  to  carry 
out  prior  studies  to  determine  whether  each  of  the  levels  of  the  experi- 
ment as  defined  in  physically  measurable  terms  is  meaningful.  These 
antecedent  studios  would  have  the  goal  of  determining  which  of  these 
physically  defined  conditions  are  different  in  terms  of  the  pilots' 
performance. 

It  is  evident  that  many  character i sties  of  a simulator  in  which  wo 
may  have  a valid  interest  as  to  their  training  effectiveness  may  be 
defined  in  physical  units  which  can  be  differentiated  through  use  of  the 
appropriate  metric  but  which  cannot  be  differentiated  by  the  operating 
pilot  at  the  level  of  his  perception  and/or  performance.  This  Is  the 
classical  psychophysical  problem  in  which  Intervals  on  the  psychological 
scale  bear  a relationship  to  the  intervals  of  the  physical  scale,  but 
not  necessarily  one  which  is  linear  or  even  monotonic.  In  training 
effectiveness  research  the  levels  of  a variable  being  Investigated  must 
be  set  based  upon  Intervals  differentiated  in  terms  of  performance  rather 
than  physical  units.  The  determination  of  the  performance  intervals  is 
prerequisite  to  transfer  of  training  research  for  many  variables  of 
interest.  To  illustrate  further  the  importance  of  the  psychophysical 
relationship  we  ma\  consider  the  fidelity  of  motion  variable  in  training 
simulator  research.  The  degree  of  fidelity  of  motion  may  te  defined  In 
physicaily  measurable  terms  in  a variety  of  ways  which  could  be  varied 
systematically  to  determine  its  effect  upon  transfer  ov  training. 

Parenthetically,  in  choosing  the  physical  expression  of  a variable  cne 


10 


should  bo  so  looted  which  is  meaningful  and  useful  for  the  simulator  de- 
sign engineer.  The  fidelity  of  motion  variable  may  be  expressed  In  terms 
of  the  frequency  response  of  the  motion  platform  to  some  driving  function 
which  might  be  varied  over  a range  of  physical  values  for  experimental 
purposes.  However,  In  this  case,  as  Is  true  in  many  others,  there  Is 
virtually  no  background  Information  to  guide  the  experimenter  In  setting 
levels  of  the  experimental  variable.  As  stated  earlier.  It  Is  necessary, 
through  essentially  a psychophysical  procedure,  to  establish  dlscrlmlnab le 
Intervals  of  the  variable  before  considering  further  experimentation. 

What  must  be  established  ore  the  physical  intervals  of  the  variable  which 
are  different  for  the  operator  In  terms  of  his  performance.  Like  the 
fidelity  of  motion  variable,  a great  many  variables  of  'research  interest 
require  establishing  the  psychophysical  scale  before  experimentation 
can  proceed. 


What  Is  proposed  Is  that  prior  to  the  conduct  of  transfer  of  train- 
ing experiments.  It  Is  necessary  to  establish,  for  many  experimental 
variables,  that  the  conditions  or  levels  of  any  variable  being  examined 
are  different  In  terms  of  operator  performance.  Conditions  or  levels’ 
set  up  on  the  basis  of  the  physical  measurement  of  the  variable  may  or 
may  not  be  different  in  any  way  meaningful  or  even  perceptible  to  the 
pilot.  Those  levels  of  the  experimental  variable  which  do  not  differ 
wfth  respect  to  performance  could  not  be  expected  to  differentiate  with 
respect  to  transfer  of  training.  It  Is  to  be  emphasized  that  establish- 
ing the  limits  of  the  physical  scale  within  which  performance  Is  equi- 
valent establishes  an  Interval  on  the  performance  scale  and  that  levels 
of  the  experimental  variables  must  be  set  on  the  basis  of  this  scale. 


Once  the  ASUPT  has  been  established  as  the  criterion  system  a less 
costly  method  than  classical  transfer  can  be  used  for  obtaining  infor- 
mation about  the  effectiveness  of  simplified  systems.  The  method  rests 
on  the  assumptions  that  transfer  of  training  from  the  criterion  ASUPT  Is 
positive  and  high  and  that  a modification  of  ASUPT,  even  though  different 
from  the  criterion  ASUPT  In  Its  physical  metric,  when  equivalent  to  It  In 
terms  of  pilot  performance  will  provide  equally  positive  and  high 
transfer.  Given  these  assumptions  many  simulator  characteristics  which 
are  Important  to  Investigate  from  a cost-effectiveness  stand-point  may 
be  studied  without  conducting  formal  transfer  of  training  experiments. 


The  minimal  system  which  is  equivalent  ro  the  criterion  ASUPT  with 
respect  to  any  simulator  characteristic  Is  just  that  - the  minimal 
essential  physical  system  which  Is  equivalent  In  terms  of  performance 
to  the  more  complex  one.  it  Is  assumed  that  transfer  to  the  aircraft 
would  be  as  great  using  that  system  as  with  the  criterion  system.  How- 
ever, other  configurations  which  could  be  quite  unlike  the  criterion 
ASUPT  in  some  of  its  characteristics  might  be  be+ter  training  configu- 
rations. That  Is,  the  trainer  may  be  modified  in  some  of  Its  character- 
istics or  additions  made  to  It  to  enhance  Its  efficiency  as  a training 
device. 
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The  point  just  made  leads  to  making  the  distinction  between  effi- 
cient training  and  effective  training.  Efficient  training  Is  that  which 
brings  the  student  to  the  Immediate  training  goal  most  economically  In 
the  least  training  time.  If  the  skill  attained  upon  reaching  the  Imme- 
diate training  goal  can  be  applied  In  a larger  training  objective  the 
Immediate  goal  training  Is  effective  In  terms  of  the  larger  goal.  For 
example,  !f  the  cues  used  In  approach  and  landing  can  be  taught  more 
quickly  usl  g U highly  abstract  display  we  have  an  efficient  training 
technique.  It  will  be  an  effective  technique  only  to  the  extent  that  the 
skills  acquired  by  that  technique  can  be  applied  In  the  larger  more 
realistic  situation,  l.e.,  positive  transfer  takes  place. 

We  may  assume  that  any  training  which  transfers  positively  to  the 
ASUPT  criterion  system  Is  effective  training  for  the  aircraft.  Or  we  may 
not  accept  as  effective  any  training  method,  procedure  or  equipment  until 
It  has  been  demonstrated  that  positive  transfer  to  the  aircraft  Is  the 
case.  The  choice  Is  simple  but  significant  with  respect  to  cost.  The 
adoption  of  the  ASUPT  as  the  criterion  system  for  Investigation  of  the 
most  efficient  training  methods  and  simulator  utilization  Is  believed  to 
involve  small  risk.  In  determining  the  minimally  essential  character- 
istics of  the  training  simulator,  the  use  of  performance  equivalence 
methods  to  establish  the  least  common  denominator  which  Is  equivalent  to 
the  ASUPT  criterion  system  appears  logically  sound.  The  experimental 
data  to  support  it  are  sparse  although  experience  with  simulator  con- 
figurations over  the  years  lends  It  credence.  It  only  needs  to  be  re- 
called that  all  manner  of  simulator  configurations  have  been  used  with 
positive  results.  What  Is  needed  Is  the  system I zat I on  of  the  method 
tor  varying  those  configurations  to  determine  which  we  might  settle  upon 
for  use.  The  transfer  of  the  training  from  that  configuration  seems 
assured.  Studies  designed  to  develop  and  test  such  an  approach  are 
proposed  In  later  sections  of  this  report. 

4.2  Performance  Measurement 


In  assessing  the  rate  of  acquisition  of  skill  or  the  final  level  of 
skill  of  an  operator  of  a complex  system  such  as  an  aircraft,  performance 
may  be  evaluated  at  two  points  in  the  system.  These  are  at  the  system 
output  point  through  such  measures  as  heading,  airspeed  and  altitude  and 
at  the  control  Input  point  through. measurement  of  control  movements. 

These  measurement  points  <MP)  are  shown  in  Figure  4,1. 

It  seems  a reasonable  assumption  that  an  efficient  training  system 
would  astab-fsh  in  the  trainee  quickly  and  economically  the  control 
Input  behaviors  of  the  experienced  and  skilled  operator  appropriate  to 
bringing  about  the  specified  system  output.  Training  may  be  thought  of 
as  shaping  this  control  output  behavior.  Knowledge  of  the  level  of 
this  behavior  as  training  progresses  would  be  useful  to  the  Instructor 
in  guiding  the  student. 
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Figure  4.1  Performance  Measurement  Points  (MP)  In  the 
Man/Mach  I ne  System 


When  all  training  Is  given  In  the  aircraft,  I.e.,  the  final  criterion 
system,  concern  with  the  control  Input  as  an  Index  of  skill  level  as 
opposed  to  system  output  Is  a less  Important  matter.  System  outputs  are 
available,  have  been  used  as  measures  over  a long  period  of  time  and  a 
good  deal  Is  known  about  them.  (See  Demareo,  et  al.,  1964)  Concern  with 
the  operator  output  parameters  begins  when  the  desire  Is  to  train  In  a 
simulator  the  skills  required  for  precise  control  of  a dynamic  system 
such  as  an  aircraft.  It  Is  then  that  system  output  measures  may  be  mis- 
\ leading  since  the  human  operator  can  adapt  his  control  behavior  to  bring 

about  the  same  system  output  over  a broad  range  of  dtffereril  systems. 

In  this  case,  the  simulator  may  shape  control  Input  behavior  not  directly 
transferable  to  control  of  Ihe  aircraft  If  the  simulator  differs  In  the 
criterion  system  In  significant  ways.  It  may  be  possible  for  the  operator 
to  control  the  system  such  that  the  correct  system  output  Is  obtained  while 
at  the  same  time  requiring  quite  different  control"  Inputs  - Inputs  which 
would  then  be  Inappropriate  to  the  criterion  system  with  resu'tlng  reduced, 
or  even  negative  transfer  of  training. 

The  emphasis  here  on  measurement  of  operator  output  behavior  at  the 
control  Input  point  Is  Its  use  in  the  performance  equivalence  approach  to 
I the  Investigation  of  the  necessary  characteristics  of  training  simulators, 

I With  this  approach  the  experienced  pilot  Is  a ’’cal Ibrated"  control  element 

| being  observed  to  determine  whether  his  control  output  behavior  changes 

1 with  prescribed  changes  In  the  system  which  he  Is  controlling  when 

f system  performance  Is  maintained  at  a prescribed  level.  What  Is  sought 

5 Is  a measure  which  Is  a reliable  description  of  the  pilot’s  control 
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behavior  and  fs  sensitive  to  changes  In  task  conditions  such  as  tho 
dynamics  of  the  system  being  operated  or  changes  In  the  sources  of  his 
Information  for  controlling  the  system.  An  example  of  the  latter  Is  a 
change  In  the  dimensions  of  movement  of  control  of  the  motion  platform 
In  an  aircraft  simulator. 

The  behavior  of  the  human  controller  In  a complex  closed- loop  system 
is  exhibited  In  a complex  time  varying  output  through  the  controls  of  the 
system.  A simple,  direct,  and  preferably  "on-line"  summarization  of  this 
output  Is  required.  Previous  work  has  shown  that  such  measures  as  the 
power  density  spectrum,  the  breakpoint  f rer..  ^ncy  of  the  transform  of  that 
spectrum  and  the  percentage  of  power  In  sjljcted  portions  of  the  spectrum 
are  summary  measures  which  reflect  meanln’/ul  changes  In  the  operator’s 
behavior  with  changes  !n  the  conditions  o\  the  task  and  may  be  useful 
measures.  (See  Norman,  1973  and  Matheny  et  at.,  1974)  The  relative  re- 
liability of  such  summary  measures  and  their  Inter-relationships  need  to 
be  established.  What  is  required  Is  an  exaninatlon  and  analysis  of 
pilot  control  output  behavior  In  the  complex  control  situation  across  a 
range  of  conditions  of  extern*1  disturbances  to  the  system  and  of  re- 
quirements for  control,  i.e.,  maneuvers  or  "asks.  Systems  v<hlch  have  the 
capability  for  experimental  manipulation  of  variables  for  studying  this 
behavior  and  providing  answers  to  practical  questions  regarding  simulation 
requirements  are  ASUPT,  T-40  and  T-4G  simulators,  and  the  Instrumented 
T-37  aircraft.  The  use  of  these  devices  In  developing  such  performance 
measures  and  In  the  overall  research  program  Is  discussed  In  Section  5.0. 


5.0  RESEARCH  PROGRAM 


5.1  Overall  Plan 


The  major  elements  In  an  overall  plan  for  a research  program  In  under- 
graduate pilot  training  are  shown  In  Figure  5.1.  This  plan  Includes  as  a 
first  phase  development  of  procedures,  methods  and  measurement  techniques 
for  use  of  the  research  equipment.  The  second  phase  Includes  carrying  out 
screening  experiments  and  more  formal  Investigations,  The  first  stage 
serves  to  provide  a technological  base  for  experimentation  and  Is  funda- 
mental to  research  In  the  program.  It  is  designed  also  to  produce  data 
applicable  to  other  training  research  programs  and  to  training  research 
device  procurements. 

The  research  phase  of  the  program  fs  divided  into  Investigations  of 
training  methodology  and  of  training  simulator  requirements.  Each  of 
these  areas  Is  further  divided  Into  a screening  and  a validating  phase. 

The  screening  phase  Is  designed  to  narrow  down  the  multitude  of  variables 
in  a systematic  way,  particularly  with  respect  to  the  simulate*  re- 
quirements research  area  prior  to  more  Intensive  formal  experiments. 

It  should  be  mentioned  that  training  methodology  and  simulator 
requirements  are  not  mutually  exclusive  areas  of  research.  Certain 
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'raining  methodologies  will  require  the  use  of  particular  simulator 
characteristics.  Conversely,  certain  simulator  requirements  will  require 
the  utilization  of  certain  :r»s1hcdologies.  It  should  be  noted  also  that 
the  results  of  screening  experiments  may  form  the  basis  for  recommenda- 
tions for  improving  training  effectiveness  without  going  through  the 
more  formal  experimental  stage. 

The  research  plan  provides  for  the  investigation  and  test  of  certain 
less  well  used  Investigative  methods  with  the  goal  of  determining  the 
utility  of  more  economical  experimental  approaches  and  designs. 

The  various  elements  In  the  overall  research  plan  are  discussed  in 
the  sections  to  follow. 

5.2  Establishing  a Technological  Base 

Establishing  a technological  base  means  simply  that  the  development 
and  tost  of  procedures,  measurement  techniques,  equipment  checks  and 
the  like  are  necessary  antecedents  to  carrying  out  a research  program. 

The  techno I ©g I cal  base  discussed  here  extends  beyond  this  since  It 
recommends  a somewhat  different  approach  by  proposing  to  establish  the 
ASUPT  as  a criterion  device  for  the  conduct  of  Investigations  and 
experiments.  This  phase  of  the  program  Is  intended  to  provide  the  data 
for  the  characteristics  of  a training  simulator.  This  performance 
equivalence  approach  was  discussed  In  Section  4.G, 

It  should  be  pointed  out  also  with  respect  to  establishing  the 
t chnological  base  that  the  working  out  of  procedures,  measurement 
techniques  and  fhe  like  are  proposed  to  be  accomplished  using  specifi- 
cally designated  experimental  variables  as  the  vehicle  for  developing 
the  procedures  and  techniques.  The  working  through  of  such  an  experi- 
ment is  Intended  to  provide  practical  Information  for  the  technological 
base  as  well  as  to  provide  useful  data  on  the  variables  involved, 
however,  limited  or  preliminary.  This  experiment  is  proposed  to  be  one 
In  the  simulator  requirement's  research  area. 

5.2.1  Equipment  Famil lari zat I on 


The  Information  to  be  gained  and  procedures  to  be  established  under 
this  category  are  concerned  principally  with  experimental  equipment 
characteristics.  Here  the  research  personnel  will  gain  familiarity 
with  the  problems  and  procedures  In  configuring  the  equipment. 
Generally,  the  order  of  experimental  equipment  configuration  is  deter- 
mined by  the  experimental  paradigm  adopted  by  the  researcher.  However, 
in  those  experiments  dealing  with  simulator  requirements  using  the 
performance  equivalence  paradigm  the  order  of  experimental  conditions 
may  be  set  up  to  be  most  efficient  In  terms  of  equipment  change  rather 
than  overall  experimental  paradigm.  This  holds  since  experienced 
pilots  will  be  used  as  subjects  and  will  be  trained  to  specified  levels 
of  system  output  under  given  experimental  conditions.  Under  this 


experimental  method  it  is  not  critic'll  that  the  conditions  be  ordered 
In  any  particular  manner  with  respect  to  subject  learning.  It  is 
possible  then  to  order  the  experimental  conditions  in  terms  of  ease  of 
transitioning  from  ons  equipment  conf iguratlon  to  another  rather  than 
in  accordance  with  some  overall  experimental  paradigm.  Information 
obtained  as  to  the  most  economical  way  of  changing  from  one  equipment 
condition  to  another  can  be  obtained  during  the  early  equipment  familiar- 
ization phase  and  will  prove  a valuable  time  saver  later. 

These  preliminary  studies  and  Investigations  will  afford  the  oppor- 
tunity for  oblainlng  data  on  the  consistency  and  reliability  of  the 
equipment  under  continued  experimental  use.  idiosyncrasies  within  the 
system  can  be  identified  so  that  they  may  be  taken  into  account  in  planning 
future  experiments. 

A necessary  concern  Is  with  establishing  calibration  procedures  for 
the  important  and  necessary  parameters  within  the  system  during  experi- 
mentation. The  type  and  frequency  of  calibration  necessary  to  keep  the 
equipment  functioning  at  a standard  level  for  experimental  purposes  must 
be  established  during  this  phase  and  provides  data  for  the  technological 
base. 

An  Important  function  to  be  accomplished  during  this  phase  Is  the 
determination  of  how  the  experimenters  and  Instructors  will  function  at 
the  ASUPT  console  or  Instructors*  stations  Ir  carrying  out  the  experi- 
ments. Since  the  two  instructor  stations  d!  fer  in  terms  of  the  Infor- 
mation available  to  the  experimenter  or  Instructor,  the  procedures  and 
practices  f ' providing  instructions  to  the  subjects  need  to  be  worked 
out  and  function  smoothly. 

5.2.2  System  Measures 

These  measures  are  those  referred  to  as  M2  in  Figure  4.2  and  roflect 
the  performance  of  the  total  man-machine  system.  In  aircraft  they  are 
such  parameters  as  airspeed,  heading,  altitude,  and  pitch  or  roli  angle. 

It  is  necessary  that  these  be  developed  and  specified  for  each  of  the 
research  devices  such  as  ASUPT,  the  instrumented  airplane  and  the  T-4C 
and  T-4G  trainers.  The  specification  of  these  measures  is  Important 
throughout  all  phases  of  experimentation  although  their  use  differs 
between  the  training  methodology  and  the  simulator  requirements  areas. 

In  the  areas  .of  precision  of  aircraft  control  useful  and  reliable 
measures  have  been  developed  for  most  maneuvers  both  in  the  simulators 
and  In  the  aircraft.  Criteria  against  which  performance  may  be  measured 
can  also  be  rather  readily  established  for  carrying  out  the  required  task 
procedures. 

System  output  measures  are  being  established  by  the  Human  Resources 
Laboratory,  Flying  Training  Division,  through  contract  and  HRL/FT 
personnel  efforts.  The  HRL/FT  effort  has  bsen  summarized  by  Waag  (1974) 

In  which  he  points  out  that  the  major  emphasis  In  the  present  performance 
measurement  development  Is  toward  developing  those  measures  which  reflect 


performance  in  the  simulator*  Ue,,  ASUPT.  Emphasis  is  placed  upon  these 
performance  measures  since  it  is  assumed  that  performance  in  the  simulator 
will  be  positively  related  to  performance  in  the  aircraft  and  It  is, 
therefore,  possible  to  use  the  simulator  as  a criterion  device  for  in- 
vestigating hardware  configurations  and  training  strategies.  The  HRL/FT 
in-house  work  is  aiso  directed  toward  defining  the  objectives  for  any 
particular  maneuver  or  sequence  of  training  tasks  and  +~  dove  I op  measures 
for  thos-a  particular  criteria,  A major  effort  is  being  directed  toward 
developing  measures  which  will  determine  the  degree  to  which  the  criterion 
objectives  are  met;  reflect  only  the  salient  characteristics  of  perfor- 
mance; are  useful  by  the  student  and  the  instructor  pilot;  and  are 
generated  on  a real-time  basis  so  that  feedback  may  be  immediate. 


The  systems  measurement  work  being  carried  out  by  HRL/FT  personnel 
provides  a sound  basis  for  selection  of  system  measures  for  any  given 
experimental  Investigation.  Life  Sciences'  efforts  have,  therefore, 
been  directed  toward  the  development  cf  operator  performance  measures 
ratner  than  system  measures.  It  should  be  noted  here,  however,  that 
HRL/FT  efforts  have  not  been  limited  solely  to  systems  measures  but  have 
aiso  been  concerned  with  operator  output  measurement. 


5,2.3  Operator  Performance  Measures  - T-40  Experiment 


Operator  performance  measurement  was  discussed  in  Section  4,2  in 
connection  with  the  performance  equivalence  experimental  paradigm.  These 
measures  are  taken  at  the  controls  operated  by  the  pilot  and  reflect  his 
inputs  into  the  system.  They  reflect  his  direct  response  to  the  stimuli 
presented  to  him  by  the  system.  They  are  distinguished  from  system 
performance  records  which  reflect  his  response  plus  the  variability  in 
the  system  through  which  he  is  operating. 


investigations  in  the  T'40  training  simulator  can  be  designed  to 
examine  different  procedures  for  summarizing  the  time  varying  operator 
output  i.nto  the  controls,  to  examine  Inter-’ individual  differences  in 
these  summary  measures  and  to  determine  the  stability  or  reliability  of 
these  measures  over  time.  This  device  cojpied  with  the  ADACS  data  re- 
cording system  constitutes  a most  suitable  device  for  preliminary  In- 
vestigations of  these  questions. 


As  a general  condition  of  the  experiments  in  which  operator  output 
measures  are  being  examined,  the  pilot  must  control  the  system  to  a 
prescribed  lovel  of  system  performance  at  which  time  his  control  output 
behavior  is  recorded,  it  Is  necessary  to  standardize  system  performance 
in  order  that  changes  introduced  into  the  system  being  controlled  may  be 
reflected  in  the  operator's  control  output.  Control  output  behavior  is 
than  compared  across  changes  introduced  into  the  system  or  ihe  task 
being  carried  out.  When  such  changes  do  not  r&sulf  Jo  changes  in  the 
experienced  operator's  control  behavior,  the  conditions  being  studied 
are  concluded  to  be  behavioral iy  equivalent. 


4.  Sys  output  measures  are  heading,  altitude,  airspeed, 
bank  angle  and  pitch  angle. 

5.  Operator  output  measures  are  the  subject  of  the  Investigation 
and  analysis  is  par formed  on  the  recorded  output  of  fore-and- 
aft  and  lateral  stick  and  of  throttle. 

6.  Conditions  tabled  above  are  for  a single  subject.  The  order 
of  trial  conditions  may  be  varied  from  subject  to  subject  to 
obtain  an  estimate  of  order  effect.  However,  the  subject  . 
practices  each  condition  to  criterion  level  so  this  effect 
should  not  be  significant.  It  is  desirable  to  space  trials 
and  conditions  to  avoid  fatigue  and  boredom. 

7.  Subjects  repeat  the  conditions  after  one  day  and  again  after 
one  week. 

In  this  expei iment  the  measures  of  particular  interest  are  the  pilot’s 
control  output  measures  at  the  rime  he  has  reached  specified  levels  of 
performance  on  the  system  output  measures.  The  control  measures  arb 
derived  from  fore-and-aft  and  lateral  movements  of  the  stick  and  move- 
ments of  the  throttle.  These  movements  may  be  summarized  in  a number 
of  ways.  What  is  sought  Is  a simple,  on-line  summarization  which  re- 
flects reliably  the  pilot’s  performance  and  Is  sensitive  to  changes  in 
it.  It  should  reflect  changes  with  practice  and  level  off  at  the  same 
point  at  asymptote.  Those  measures  which  are  proposed  to  be  examined 
are:  Stick  force  (mean  and  mean  square  value):  stick  Z score  and 
throttle  Z so:re  as  developed  by  Dr.  W.  Waag  of  HRL/FT  for  the  ASUPT 
program  (the  iium  cf  the  squared  differences  between  present  stick  vector 
position  and  last  stick  vector  position):  stick  fore-and-aft  and 

lateral  position  computed  In  the  same  manner  as  stick  Z:  the  proportion 

of  operator  output  power  below  6 radians  and  above  2,  termed  cross-over 
power  (see  Norman,  1974):  and  transforms  of  the  pilot  output  power 
spectrum  which  allow  determination  of  the  upper  breakpoint  frequency  of 
the  pilot’s  output  (see  Matheny  et  al.,  1974),  It  is  also  proposed  that 
the  power  in  narrow  frequency  bands  around  6 radians  will  be  examined 
for  the  shift  In  power  as  a function  of  experimental  conditions.  The 
results  of  these  analyses  will  be  used  to  guide  the  development  of 
operator  performance  records  in  the  ASUPT  and  the  instrumented  aircraft. 

5.2.4  Operator  Performance  Measures  - ASUPT 

It  is  proposed  as  a part  of  the  collection  of  technological  base 
Information  that  an  initial  experiment  be  carried  out  in  ASUPT.  The 
principle  purpose  is  the  gathering  of  Information  about  opera  for 
performance  measures  and  equipment  procedures.  However,  at  the  same 
time.  it  may  provide  information  and  Insights  about  the  effects  of  some 
relevant  variables.  This  experiment  is  proposed  to  repeat  the 
essentials  of  the  T-40  experiment  described  in  Section  5.2.3  with  an 


additional  vaHabie  tanned  "visual  display".  Tha  selection  of  conditions 
will  be  guided  by  the  results  of  the  T-40  experiment  ortce  they  are 
available.  However,  for  planning  purposes,,  It  Is  proposed  that  the 
variables  and  levels  listed'  In  Table  5,2  be  t?ose  to  be  Investigated. 

This  experiment  becomes  one  which  essentially  examines  the  question  of 
the  Interaction  between  the  motion  ano  the  visual  display  In  the 
simulator. 


Table  5.2 

Experimental  Conditions  for  Investigating  Operator 
Performance  Measures  In  ASUPT 


CONDITION 


VARIABLES 


Turbulence  Platform  Motion  Visual  Display  G-Seat 


1 

Low 

On 

On 

Off 

2 

High 

On 

On 

0'f 

3 

Low 

On 

Off 

Off 

4 

High 

On 

Off 

Off 

5 

Law 

Off 

On 

Off 

6 

High 

Off 

On 

Off 

7 

Low 

Off 

Off 

Off 

8 

High 

Off 

Oft 

Off 

9 

Low 

On 

On 

On 

10 

High 

On 

On 

Or. 

H 

Low 

On 

Off 

On 

12 

High 

On 

Off 

On 

‘3 

Low 

Off 

On 

On 

14 

High 

Off 

On 

On 

15 

Low 

Off 

Off 

On 

16 

High 

Off 

Off 

On 

The 

general  experimental 

paradigm 

*s  the  same  as 

In  T-40  expo 

In  that  experienced  pilots  would  practice  each  experimental  condition  to 
a specified  level  at  which  time  control  outputs  would  be  recorded  over 
a two  mfmrto  trial  for  analysis.  Although  the  conditions  are  the  same 
as  those  for  the  T~40  experiment  listed  in  Section  5,2.3,  they  are  re- 
peated here  ?n  the  Interest  of  completeness. 

1.  Trials  are  two  minutes  In  length, 

2.  Subject  pilots  are  to  be  experienced  Instructor  pilots. 

3.  Ccndtions  are  to  be  carried  out  fc**  both  Straight  and 
Uve!  end  30°  Bank  maneuvers. 
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4.  Subject  will  practice  the  maneuver  under  each  condition 
until  he  reaches  criterion  performance  on  the  system 
specified  output  measures* 

5.  System  output  measures  are  heading,  altitude,  airspeed, 
bank  angle  and  pitch  angle. 

6.  Operator  output  measures  are  the  subject  of  the  Investigation 
and  analysts  Is  performed  on  the  recorded  output  of  fore- 
and-aft  and  lateral  stick  and  of  throttle,, 

7.  Conditions  tabled  above  are  for  a single  subject.  The 
order  of  trial  conditions  may  he  varied  from  subject  to 
subject  to  obtain  an  estimate  of  order  effect.  However, 
the  subject  practices  each  condition  to  criterion  level 

so  this  effect  should  not  be  •;! f leant.  It  is  desirable 

to  space  trials  and  conditions  to  avoid  fatigue  and  boredom. 

8.  Subjects  repeat  the  conditions  after  one  day  and  again 
after  one  week. 

The  contact  visual  display  to  be  used  Is  the  full  width  of  view 
present  in  the  ASUPT.  The  content  of  the  display  should  consist  of  a 
definitive  horizon  line  with  distinguishing  features  on  the  horizon  as 
external  referents  for  a heading  index.  A ground  plane  selected  from 
those  available  In  the  visual  display  data  bank  judged  by  instructor 
pilots  to  be  representative  of  the  local  terrain  will  be  used. 

5.2.5  Operator  Performance  Measures  in  the  Instrumented  Aircraft 


The  need  for  operator  performance  data  obtained  lr«  the  aircraft  was 
indicated  in  the  discussion  of  methodological  considerations,  Section 
4.0.  Operator  performance  output  data  in  the  ai rereft  being  simulated 
is  proposed  t->  be  used  to  establish  a data  base  for  comparing  performance 
In  the  ASUPT  to  that  In  the  aircraft.  Data  reflecting  how  the  operator 
responds  in  controlling  the  aircraft  under  representative  maneuvers  and 
conditions  are  fundamental  to  developing  the  performance  equivalence 
approach  to  determining  the  necessary  and  sufficient  simulator  require- 
ments. By  way  of  review,  the  performance  equivalence  approach  postulates 
tha4'  two  systems  are  equivalent  with  respect  to  performance  if  operator 
output  performance  is  the  same  for  both  systems  when  those  systems  are 
being  controlled  to  the  same  levels  of  system  output. 

The  existence  of  the  ASUPT  and  an  instrumented  T-37  aircraft 
presents  an  unusual  opportunity  to  develop  and  test  more  objective 
methods  for  stating  and  evaluating  training  simulator  requirements.  The 
unique  capability  of  the  ASUPT  of  being  varied  in  its  characteristics, 
coupled  with  the  capability  for  obtaining  reliable  measures  in  the  air- 
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craft  being  simulated,  makes  possible  research  Into  the  perennial 
problem  of  how  phychologica!  equivalence  can  be  established  between  the 
•simulator  and  the  vehicle  being  simulated  in  terms  which  are  quanti- 
tative and  useful  to  the  simulator  design  engineer. 

As  indicated  in  Section  4,0  it  is  proposed  that  pilot  performance 
be  recorded  in  the  instrumented  T-37  aircraft  across  a representative 
set  of  maneuvers  and  conditions  and  a data  base  be  established  with  re- 
spect to  this  performance.  It  is  proposed  that  summary  measures  of  the 
time  varying  output  to  the  control  by  the  pilot  be  obtained  and  that 
these  measures  be  gathered  using  highly  experienced  pilots  flying  each 
maneuver  to  a specified  criteria.  Data  will  be  collected  such  that  the 
population  statistics  of  mean  and  variance  in  performance  may  be 
estimated.  These  statistical  values  will  represent  the  population 
values  for  the  parameters  under  investigation.  The  operator  control 
performances  of  interest  are  proposed  to  be  the  inputs  into  the  stick, 
both  fore-and-aft  and  lateral,  and  into  the  throttle. 

A further  use  of  the  data  obtained  in  the  instrumented  T-37  air- 
craft is  Its  use  In  summarizing  the  kind  of  control  input  behavior 
exercised  by  experienced  instructor  pilots  In  the  T-37.  These  data  can 
be  considered  to  be  the  criterion  performances  toward  which  the  student’s 
behavior  is  to  be  shaped  during  the  training  process.  Continuous 
monitoring  of  the  trainee's  control  inputs  will  provide  diagnostic  infor- 
mation which  may  be  fed  back  to  the  student  to  influence  his  rate  of 
learning. 

In  carrying  out  the  Instrumented  aircraft  flights  experienced 
instructor  pilots  will  perform  to  specified  system  output  criteria  during 
certain  designated  maneuvers.  System  performance  measures  will  be  re- 
corded along  with  the  operator's  control  output  performance.  System  per- 
formance measures  are  to  be  the  same  as  those  recorded  in  the  T-40  and 
ASUPT  experiments  and  are  heading,  airspeed,  altitude,  pitch  angle  and 
roll  angle.  The  records  of  control  movements  during  these  performances 
will  be  summarized  in  the  same  manner  as  that  developed  from  experimen- 
tation in  the  T-40  and  ASUPT.  The  measurement  parameters  to  be  recorded 
In  the  aircraft  with  the  ranges,  accuracies  and  sample  rates  are  given 
in  Table  5.3. 

The  linear  accelerometers  listed  in  Table  5,3  are  proposed  to  be 
mounted  orthogonal  to  each  other  at  a position  on  the  pilot's  seat  as 
close  to  the  pilot’s  head  as  posable. 

The  conditions  under  which  data  are  to  be  collected  are  given  In 
Table  5.4.  This  table  must  be  Interpreted  in  connection  with  the  flight 
pattern  and  sequence  for  recording  trials  depicted  in  Figures  5.2  and 
5.3  respectively. 


Table  5.3 


Parameters  Proposed  to  be  Recorded  in 
the  Instrumented  T-37  Aircraft 


Parame1«.«r 

Range 

Accuracy 

Sample 

Rate/Sec 

Elev  Stick  Force 

0-30  lb 

+.1  lb 

too 

Aileron  Stick  Force 

0-20  lb 

±.l  lb 

100  i 

Rudder  Force 

0-30  lb 

±*  1 ib 

100 

Elevator  Position 

-16°  - +24* 

.5* 

100 

Al leron  Position 

±15° 

.5* 

100  j 

Rudder  Position 

±24* 

.5* 

ioo  ! 

1 

I 

Throttle  Position 

Full 

.5* 

1 

100  j 

Low  Altitude 

0— 5m* 

o 

cs 

-H 

10  ! 

High  Altitude 

0-25m 

+50  • 

io 

G’s 

-IG  - +5G 

±.  IG 

10 

Heading 

0-360* 

±1* 

10 

Yaw  Rate 

±70*/Sec 

+I*/Sec 

10  I 

j 

Trim  Tab  Position 

On  - Off 

ioo  ; 

Pitch  Rate 

±90 “/Sec 

l°/Sec 

:o 

Airspeed 

0-300K 

+ IK 

ioo 

Roll  Rate 

±I00*/Sec 

+I°/Sec 

10 

Roll  Angle 

0-260* 

+ 1° 

too 

Pitch  Angle 

0-360* 

+ 1° 

100 

l JC**~^,LL*f£ZtLS  rndfi,  — T.'lV?,  - »*.  ti  t ^>•■-1  --»' 
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Table  5.4 


Conditions  and  Sequencing  Data  Collection  Sorties 
In  Instrumented  T-37  Aircraft  Per  Pilot 


Sortie 

Trial  Order* 

A/S 

Inst/Contac 

1 

A 

200 

Inst 

2 

B 

200 

Inst 

3 

A 

100 

Inst 

4 

B 

100 

Inst 

5 

A 

200 

Contact 

5 

B 

200 

Contact 

7 

A 

100 

Contact 

8 

B 

too 

Contact 

* For  trial  order  see  Figure  5.3 


In  the  Instrumented  aircraft  it  Is  proposed  that  data  be  collected  on 
the  maneuvers  cf  climb,  descent,  straight  and  level  and  30  deg  turns. 

Ourinq  these  maneuvers  tolerances  will  be  held  as  closely  as  possible  to 
the  standard  determined  for  that  aircraft.  Data  will  be  recorded  during 
the  periods  shown  In  Figure  5.2.  This  scenario  assumes  that  flight 
data  will  be  collected  with  the  aircraft  being  flown  from  Wrlght-Patterson 
AFB  to  a test  flight  area  and  that  return  from  that  area  may  be  made  at 
low  altitude  during  which  additional  data  on  straight  and  level  may  be 
collected.  It  Is  proposed  that  the  maneuvers  be  carried  out  both  on 
Instruments  and  contact  with  the  contact  maneuvers  being  flown  following 
Initial  analysis  of  the  Instrument  data. 

A detailed  scenario  of  the  date  collection  flights  In  the  Instru- 
mented T07  aircraft  is  given  In  Appendix  E.  The  manner  In  which  it  Is 
possible  to  obtain  a sample  of  sixteen  two-minute  trials  for  each  of 
the  maneuvers  of  straight  and  level  and  30  deg  bank  for  two  levels  of 
airspeed  In  counter-balanced  order  Is  indicated  In  Figure  5.3.  The 
arrangement  of  data  collection  trials  In  Figure  3.3  illustrates  the 
order  in  which  the  two  minute  trial  segments  may  be  recorded  for  both 
straight  and  level  and  30  deg  banks  during  blocks  of  two  sorties  so 
that  the  two-mlnute  trials  for  each  of  the  maneuvers  may  be  collected. 

For  example,  in  sortie  one  data  Is  collected  In  order  A,  Return  tc  Base 
from  the  test  area  may  be  flown  at  a 2,000  ft.  altitude  during  which 
three  more  two-minute  straight  and  level  trials  may  be  iecorded.  In 
sortie  2,  trials  are  flown  in  order  B,  again  with  optional  data  collection. 
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trials  recorded  during  ratum  to  base.  Each  pilot  performs  eight- 
sorties  in  obtaining  data  on  ithese  maneuvers,  both  on  Instruments  and 
contact,  i.e.,  those  shown  in  Table  5.4,  It  is  proposed  that  a 
minimum  of  five  pilots  fly  the  aircraft  data  collection  sorties  and 
repeat  them  In  the  ASUPT  simulator.  As  Indicated  earlier  comparison 
of  the  performances  in  the  simulator  and  the  aircraft  will  allow  for 
adjusting  the  simulator  to  be  equivalent  in  terms  of  pilot  performance 
to  the  T-37  aircraft. 

After  completion  of  data  collection  under  the  conditions  listed 
in  Table  5.4  data  wilt  be  collected  on  maneuvers  during  which  the  air- 
craft (s  flown  closer  to  the  limits  of  its  performance  envelope. 

These  maneuvers  are  stalls  and  loops.  It  is  proposed  that  each  of  the 
5 subject  pilots  fly  8 stalls  and  8 loops  maintaining  as  close  tolerance 
to  prescribed  limits  as  possible  while  the  parameters  listed  in  Table 
5.3  are  recorded.  It  is  estimated  that  this  will  require  two  additional 
sorties. 

In  summary  a total  of  8 sorties  per  pilot  will  be  necessary  for 
complete  data  collection  on  climbs,  descents,  straight  and  level  and 
30*  bank  turns.  An  additional  estimated  2 sorties  wii!  be  necessary 
to  obtain  data  on  stalls  and  loops. 

The  data  should  be  formatted  for  analysis  as  follows: 

1.  Nine  track  tape. 

2.  Less  than  536  characters  per  physical  racord 

3.  1600  or  800  blts/lnch 

4.  EBCDIC  (Extended  Binary  Coded  Decimal) 

5.  Even  parity 

5.2.6  ASUPT  as  a Criterion  Device 
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The  development  of  system  and  operator  performance  measures,  both 
in  the  ASUPT  and  in  the  instrumented  T-37  aircraft  are  designed  to 
provide  objective  data  for  establishing  the  ASUPT  as  a criterion  device 
for  research.  Establishing  ASUPT  as  a criterion  device  mokes  possible 
its  use  in  investigations  of  efficient  training  methods  and  the  minimum 
essential  training  simulator  requirements.  The  manner  in  which  it  is 
proposed  that  this  be  done  is  through  the  performance  equivalence 
approach  discussed  in  Section  4,0. 

Performance  data  collected  in  the  Instrumented  aircraft  provide 
the  data  base  against  which  the  performance  in  ASUPT  may  be  compare'4. 

It  is  to  be  emphasized  that  the  comparability  of  the  two  dovices  Is 
being  established  with  respect  to  the  closed-loor  dynamic  tracking 
betu-vior  required  of  the  pilot  in  controlling  the  systems.  The 
comparability  of  the  two  devices  with  respect  to  the  procedures 
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whereby  other  piloting  activities  are  carried  out  are  established  on 
the  basis  of  task  analyses  and  other  direct  comparison  methods. 

The  performance  equivalence  approach  Is  used  to  establish  the 
equivalence  of  the  simulator  to  the  aircraft  on  the  basis  of  Its  closed- 
locp  control  characteristics  sometimes  called  handling  qualities.  As 
Indicated  In  Figure  ! ^ne  present  method  for  establishing  the  simulator 
as  being  equivalent  to  the  vehicle  being  simulated  Is  the  use  of  expert 
opinion.  These  opinions  are  those  of  the  design  engineer  and  the 
experlc-’^ed  pilot.  The  aim  of  the  performance  equivalence  approach  Is 
to  provide,  more  objective  data  through  direct  comparison  of  perform- 
ance both  In  the  aircraft  and  In  the  simulator, 

6.0  RESEARCH  OUTLINES 

The  discussion  of  suggested  research  is  divided  into  that  concerned 
with  training  methodology  and  that  concerned  with  simulator  requirements. 
Both  of  these  areas  are  important  to  Improving  the  effectiveness  ot  tho 
training  program.  It  Is  postulated,  however,  that  the  area  of  training 
methodology  can  have  the  most  Immediate  and  significant  effect  upon  tne 
improvemant  of  flying  training  programs.  The  areas  of  Investigation  listed 
In  Table  5.1  are  discussed  in  this  section  with  details  of  the  experiments 
recommended  to  be  conducted. 

6. 1 Training  Simulator  Requirements  Research 

The  major  areas  of  simulator  requirements  research  judged  by  the 
panel  to  be  high  priority  Items  were  those  of  (I)  the  visual  display  to 
be  used  In  the  simulator,  (2)  the  characteristics  of  the  motion  system 
of  the  simulator  and  (3)  the  Interaction  of  the  two.  Any  research  pur- 
porting to  deal  solely  with  the  visual  or  motion  cueing  problem  must, 
due  to  the  nature  of  the  process,  be  considered  as  a study  of  the  inter- 
action between  the  motion  and  the  visual  areas.  Motion-vision  interaction 
studies  a'e  discussed  in  detail  In  paragraph  6.1.3.  The  two  areascf 
motion  and  vision  research  are  taken  up  separately  here  solely  on  the 
basis  of  the  emphasis  placed  upon  one  area  or  the  other  In  a given  re- 
search study  and  does  not  imply  a neglect  of  importance  of  the  Interaction 
effect.  Thus,  in  the  research  dealing  with  motion^  emphasis  is  on  the 
investigation  of  ? wide  spectrum  of  motion  variables  under  certain 
selective  conditions  of  visual  display.  In  the  visual  display  research 
area,  primary  emphasis  has  been  placed  upon  +he  content  of  the  visual 
display  and  its  effect  in  various  training  tasks  and  maneuvers  under  given 
conditions  of  simulator  motion.  Therefore,  in  reality  all  experiments 
proposed  to  be  undertaken  in  each  of  the  areas  provide  information  as  to 
the  interactive  effect  of  various  motion  and  visual  display  conditions. 
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6. I.!  Invest Igatf on  of  the  Content  of  the  Visual  Display 

6. 1 . 1 . 1 Background 

The  problem  of  the  content  of  the  contact  visual  display  in  the 
simulator  received  the  highest  >"**dian  rating  by  the  Training  Problem 
panel.  The  Importance  o.  the  outside  world  scene  In  the  simulator  Is 
underscored  by  the  data  collected  by  J.  Brown  (1973)  fra.  Air  Force  UPT 
students  and  instructors.  These  data  show  the  traffic  pattern  to  be 
judged  the  mosl  critical  element  of  flight  instruction,  and  the  most 
difficult  to  teach  and  to  learn.  A contact  visual  scene  is  necessary, 
of  course,  for  teaching  this  maneuver  in  the  simulator. 

The  question  as  to  the  appropriate  features  of  the  contact  world 
which  should  be  displayed  In  the  simulator  has  not  been  adequately 
answered  since  the  beginning  of  the  use  of  simulators.  Advances  in  the 
state-of-the-art  have  produced  several  types  of  complex  visual  display 
generation  and  presentation  devices.  It  Is  now  possible  to  investigate 
a broad  range  of  variables  and  to  gain  definitive  Information  as  to  the 
most  effective  display  content  for  the  training  conditions  of  Interest. 

Attempts  at  identifying  and  describing  the  visual  elements  of  a 
display  used  by  a pilot  as  cues  to  control  of  his  aircraft  have  produced 
both  highly  complex,  and  very  simple  displays.  On  the  one  hand  very 
detailed  displays  have  been  produced  In  order  to  insure  that  all  critical 
elements  are  included.  On  the  other  hand,  some  very  elemental  displays 
have  been  used  or  investigated.  Both  types  are  considered  by  their 
users  to  be  useful  for  the  purposes  intended. 

Prior  to  the  completion  of  the  present  study,  agreement  had  been 
reached  with  HRL/FT  that  the  content  of  the  visual  display  was  an  im- 
portant area  of  investigation.  It  was  also  agreed  that  during  the  period 
of  delivery,  installation,  and  calibration  of  certain  research  equipment 
investigations  would  be  undertaken  in  this  area.  It  is  necessary  to 
narrow  down  the  number  and  levels  of  the  variables  through  as  systematic 
and  valid  a procedure  as  possible  before  undertaking  ASUPT  experimentation 
This  is  necessary  since  the  number  and  combine  Hons  of  variables  possible 
of  being  investigated  become  astronomical  without  such  a screening  process 
The  use  of  eye  movement  recordings  of  experienced  pilots  flying  the  under- 
graduate pilot  training  maneuvers  was  proposed  as  a means  of  obtaining 
useful  information  about  the  objects  and  features  of  the  real  world 
scene  used  by  the  pilot,  (see  Figure  5,1)  At  the  same  time,  data  on 
the  dimensions  of  the  field  of  view  being  used  by  the  pilot  may  also  be 
obtained. 

Based  upon  (Bynum  et  al.,  1973)  experience  with  eye-movement  record- 
ings and  using  the  visual  model  for  closed- loop  cor.+rol  developed  by  Life 
Sciences,  a program  was  outlined  for  obtaining  information  relevant  to 
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the  problem  of  the  content  of  the  display.  This  program  was  Intended 
as  one  which  would  provide  useful  Information  upon  which  to  base  ASUPT 
experiments  relevant  to  visual  display  content,  and  which  could  be 
carried  out  prior  to  the  Installation  of  the  ASUPT  equipment.  To 
appreciate  how  these  preliminary  Investigations  support  ASUPT  experi- 
mentation and  to  relate  the  eye  novement  recordings  to  the  visual  model., 
a brief  description  of  the  model  and  the  part  played  by  evo  movement 
recordings  is  In  order. 

6. 1.1.2  A Visual  Model  as  a Basts  for  Hypotheses 

The  model  referred  to  has  been  described  elsewhere  In  Thielges  and 
Matheny,  (1971).  In  brief,  the  model  assumes  that  external  referents 
In  the  real  world  and  Internal  referents  fixed  to  the  vehicle  being 
controlled  may  be  projected  through  perspective  geometry  upon  a picture 
plane  perpendicular  to  the  pilot’s  I Ine-of-regard.  The  relationship 
between  the  internal  referents  and  the  external  referents  provides  Infor- 
mation with  respect  to  departures  from  desired  positions  of  the  aircraft 
in  its  six  tegraas  of  freedom  of  motion.  Different  positions  of  these 
referents  on  the  picture  plane  will  affect  differentially  the  pilot's 
ability  to  discriminate  changes  In  position  of  the  various  dimensions  of 
movement  of  the  aircraft.  For  example,  an  external  and  Internal  referent 
picked  near  the  vertical  mid-line  and  on  +he  horizon  on  the  picture 
plane- will  not  allow  the  pilot  to  discriminate  as  fine  a change  In  bank 
position  as  if  those  referents  were  picked  further  out  on  the  horizon 
from  the  mid-line.  Also,  referees  picked  near  the  horizon  do  not  allow 
for  as  fine  a discrimination  of  forward  translation  as  do  referents 
picked  closer  to  tne  aircraft  on  the  earth's  surface,  l.e.,  downward 
from  the  horizon  line  on  the  picture  plane. 

It  is  a tenet  of  the  model  that  it  Is  necessary  for  the  pilot  to 
select  Internal  and  external  referents  which  allow  him  to  make  dis- 
criminations as  to  the  changes  In  the  attitude  and  position  in  space  of 
his  aircraft.  It  is  assumed  also  that  o part  of  the  pilot's  learning 
process  is  the  identification  of  the  most  appropriate  external  and  in- 
ternal referents  for  use  in  exercising  closed- loop  control.  It  is, 
therefore,  necessary  not  only  to  identify  the  appropriate  referents  in 
the  real-world  scene  but  to  describe  the  "noise"  from  which  these  referents 
must  be  extracted  by  the  pilot. 


6. 1.1.3  Eye-Movement  Recordings 

As  mentioned,  experience  in  eye-movement  record!. .g  (Bynum  et  al., 
1973)  led  to  the  belief  that  such  recordings  could  be  used  to  identify 
external  and  internal  referents  being  used  by  the  pilot  in  the 
performance  of  the  Undergraduate  Pilot  Training  Maneuvers.  Their 
identification  could  ther.  lead  to  recommendations  as  ths  variables  to 
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be  investlgateo  more  systematically  In  the  ASUPT  compie*t  Therefore,  a 
program  for  gathering  and  analyzing  such  data  was  outlined  ?nd  re- 
commended. The  plan  for  collection  and  analysts  of  eye-movement  data  Is 
given  In  Appendix  F.  In  that  Appendix  both  content  and  field  of  view 
investigations  are  discussed  since  obtaining  information  in  the  extent 
of  visual  field  used  by  the  pilot  may  be  obtained  from  these  recordings. 
The  field  of  view  to  be  displayed  in  the  simulator  is  a practical 
oroblem  about  which  direct  information  may  be  obtained  without  added 
cost  even  though  the  panel  of  judges  did  not  rate  it  particularly  high 
as  a research  problem. 
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6* l,t,4  Expert  frontal  Approach 

\ 

It  is  appropriate  at  this  point  to  discuss  the  distinction  between\ 
the  role  played  by  the  performance  equivalence  approach  to  experimentation 
and  thd  investigation  of  the  configuration  of  the  vlsuai  display  for 
training  purposes.  As  ts  pointed  out  In  Section  4.0,  the  performance 
equivalence  approach  suggests  that  by  using  an  highly  experienced  pilot 
as  a standard  controller,  that  configuration  of  the  visual  display  which 
is  equivalent  to  the  real  world  scene  in  terms  of  performance  can  be 
arrived  at  through  systematical ly  changing  physical  variables  in  the 
ASUPT  visual  display  and  observing  performance.  So  long  as  both  system 
and  operator  performance  are  the  same  as  in  the  aircraft  the  systems  are 
judged  to  be  equivalent. 

As  indicated  in  the  discussion  of  establishing  a technological  base, 
the  first  interest  is  In  the  minimally  essential  set  of  visual  content 
conditions  which  are  equivalent  to  performance  (system  and  operator)  In 
the  aircraft.  Of  equal  importance  Is  the  determination  of  that  visual 
display  which  is  judged  by  the  performing  pilots  to  be  acceptable  as 
representative  of  the  rea!  world  scone.  The  visual  display  configuration 
will  then  be  taken  as  the  criterion  visuel  system  which  may  be  used  in 
both  performance  equivalence  and  transfer  experiments.  The  criterion 
visual  system  must  be  distinguished  from  those  configurations  which  might 
be  best  as  aids  to  training  and  from  that  configuration  which  represents 
the  absolute  minimum  sat  of  elements  for  providing  the  information  to 
the  pilot  and  makes  it  possible  for  him  to  perform  as  he  does  in  the 
aircraft. 

With  respect  to  training,  different  hypothesis  as  to  how  changes  in 
display  content  should  be  Introduced  to  produce  the  most  effective 
trainee  progress  may  be  tested  systematically  using  the  performance 
equivalent  system  3S  the  criterion  system.  For  example,  in  training  the 
student  to  discriminate  the  visual  cues  for  level  off  and  touch  down  in 
landing,  hypotheses  regarding  the  optimum  number  and  pattom  of  lines 
for  training  in  these  cues  may  be  +®rted.  instructional  assists 
entirely  foreign  to  the  real  world  scene  may  be  tested  also  to  point 
out  to  the  student  the  mosr  relevant  cues  to  his  task  of  the  moment. 
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It  will  be  recalled  that  in  the  study  by  Payne  et  al,  (1954)  it  was 
found  that  the  introduction  of  an  instructional  aid  in  the  form  of  a 
reference  guage  on  the  visual  display  enabled  the  student  to  learn  the 
proper  relationships  between  the  approach  impact  point  and  horizon  line 
when  they  were  unable  to  do  so  without  the  aid.  Such  training  aids  or 
displays  may  bring  the  student  more  quickly  to  the  point  at  which  he 
can  control  the  system,  (both  with  respect  to  system  and  operator  out- 
pvt)  as  well  as  the  experienced  pilot  when  he  (the  student)  uses  the 
training  oriented  display.  Thus,  the  training  system  is  made  more 
efficient.  However,  the  final  level  of  skill  to  be  attained  is  control 
of  the  system,  with  respect  to  both  system  and  operator  output,  using 
the  display  configuration  which  was  found  with  experienced  pilots  to  be 
equivalent  to  the  real  world  scene  If  the  training  system  Is  to  be 
effective. 


It  may  be  noted  that  at  this  point  the  assumption  may  be  made  that 
transfer  will  be  positive  and  high  from  the  equivalent  simulator  system 
sinco  performance  of  like  tasks  requires  tike  operator  performance. 
However,  in  Section  6.l.i.5  tests  of  the  relationship  of  performance 
equivalence  methodology  to  classical  transfer  experiment  are  proposed 
to  be  conducted. 


6. 1.1.5  Experimental  Investigations 


The  first  four  proposed  studies  of  the  content  of  the  visual  display 
are  shown  in  Figure  6.1.  In  brief,  they  comprise  (a)  the  establishment 
of  the  ASUPT  as  a criterion  system,  (b)  the  determination  of  the  transfer 
from  that  system  to  the  aircraft,  and  (c)  two  studies  which  are  designed 
to  test  the  performance  equivalence  approach  while  ODtaining  Information 
relevant  to  two  different  configurations  of  the  visual  display. 


In  determining  the  ASUPT  criterion  system  two  criteria  will  be  used. 
These  are  (I)  equivalent  performance  by  experienced  pilots  both  In  terms 
of  system  and  operator  output,  and  (2)  consensus  of  opinion  among  these 
experienced  pilots  that  the  visual  scene  in  ASUPT  is  subjectively 
acceptable  as  an  adequate  representation  of  the  real  world  scene.  The 
methods  and  measures  for  determining  performance  equivalence  were  dis- 
cussed in  Section  4.0. 


Three  major  variables  are  proposed  to  be  Important  in  the  ASUPT 
investigation  of  the  visual  display.  These  are  (a)  the  number  of  objects 
in  the  display;  (b)  the  placement  or  position  of  the  objects  in  the  dis- 
play; and  (c)  the  "styl Ization”  or  amount  of  detail  in  each  object. 

The  number  and  placement  of  objects  to  be  tested  car.  be  more  completely 
defined  after  analysis  of  the  eye-movement  data.  However,  certain 
guidance  for  hypothesis  formulation  can  come  from  the  visual  model 
referenced  earlier.  This  model  would  suggest  that  well  defined  ex- 
terna! reference  points  appropriately  placed  will  provide  information 
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to  the  pilot  for  control  of  the  six  dimensions  of  movement  of  his  air- 
craft* Certain  positions  of  the  referents  are  more  appropriate  to 
given  dimensions  of  movement  than  are  others. 

For  control  of  the  three  attitude  dimensions,  pitch  control  re- 
quires a reference  object  directly  ahead  of  the  aircraft  on  the  vertical 
mid-line  and  as  near  to  the  horizon  as  possible.  For  control  of  this 
dimension  alone,  only  an  identifiable  point  or  object  Is  required.  This 
obtains  as  a genera!  rule  for  the  control  of  pitch  across  ail  of  the 
maneuvers  to  be  flown*  However,  for  each  particular  maneuver  the  place- 
ment of  the  reference  object  with  respect  to  the  ground  or  sky  plane  is 
specific  to  that  maneuver  In  order  for  it  to  be  accomplished  best.  For 
straight  and  level  flight  a pitch  reference  point  ahead  of  the  aircraft 
at  the  level  of  the  horizon  line,  or  the  horizon  line  aione,  will  provide 
a reference  against  which  the  pilot  can  judge  some  internal  referent  on 
his  aircraft  in  order  to  hold  his  pitch  attitude.  For  a nose  down 
attitude  during  approach  to  landing  a reference  object  lower  on  the 
ground  piane  would  be  hypothesized  to  be  preferable. 

For  bank  control,  objects  displaced  laterally  from  the  longitudinal 
center  line  of  the  aircraft  provide  greater  amounts  of  positional  dis- 
placement per  unit  of  bank  as  they  move  further  out  from  the  center  line. 
Thus,  an  object  placed  on  the  horizon  at  the  vertical  mid-line  would 
exhibit  littie  percept ibie  movement  to  the  pilot  per  unit  of  bank.  That 
same  object  placed  30  deg  in  azimuth  from  the  vertical  mid-line  would 
provide  a greater  amount  of  displacement  for  the  same  amount  of  aircraft 
bank  ar.d  thus  enable  greater  precision  of  control. 

With  respect  to  heading  con+*-ol,  objects  on  the  horizon  in  the 
forward  viewing  area  to  whlcn  the  pilot  can  relate  an  Internal  referent 
will  provide  the  same  amounT  of  displacement  on  the  display  per  unit  of 
neoding  change.  However,  for  ease  of  scan  for  the  pilot  the  object  should 
be  directly  forward  on  the  vertical  mid-line  and  on  the  horizon. 

For  the  detection  of  longitudinal  motion  of  the  aircraft  objects 
directly  in  front  of  the  aircraft  which  appear  to  move  toward  it  are 
necessary  fer  the  best  detection  of  This  movement.  Objects  directly 
below  the  aircraft  have  a greater  perceived  displacement  while  those 
directly  forward  and  on  the  horizon  have  the  least  per  unit  of  longitu- 
dinal movement.  Therefore,  to  maximize  the  pi  lot’s  detection  of  forward 
nation  or  change  in  that  motion  objects  as  nearly  beneath  the  aircraft 
as  possible  provide  the  greatest  information  for  control  of  this 
dimension. 

The  control  of  lateral  dlsplccemenr  parallels  that  of  longitudinal. 
For  discrimination  of  displacement  alor.g  the  vertical  axis  the  pilot 
must  discriminate  changes  in  the  sizo  of  oojects  in  that  he  must  dis- 
criminate changes  in  the  relative  distance  between  one  8dge  of  an  object 
and  another  or  the  change  In  distance  between  objects. 
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The  cues  just  discussed  are  those  which  allow  the  pilot  to  control 
the  attitude  of  his  aircraft  about  Its  three  axes  and  to  discriminate 
whether  it  has  changed  its  position  in  three  dimensional  space.  In 
order  to  successfully  fly  his  aircraft,  however,  he  must  discriminate 
and  identify  objects  in  the  real  world  which  allow  him  to  direct  his 
aircraft  purposefully  from  point  to  point  or  in  the  fulfillment  of 
some  objective  or  mission.  Those  objects  which  he  must  identify  and 
use  for  direction  of  his  vehicle  are  unique  to  his  purposes  and  objective 
cf  the  jwoment.  Thus,  when  the  objective  of  the  pilot  is  to  land  the 
aircraft  on  a particular  spot  the  characteristics  of  that  spot  necessary 
for  him  to  identify  it  must  be  present  in  the  display.  Similarly,  he 
may  be  controlling  his  aircraft  in  three  dimensional  space  to  keep  it 
positioned  upon  another  aircraft  during  formation  flight.  The  relevant 
cues  peculiar  to  the  other  aircraft  which  allow  him  to  maintain  his 
position  must  be  presented  in  the  visual  display  in  order  for  the 
trainee  to  practice  In  their  use  and  become  proficient  In  formation 
flying. 

It  is  the  overall  goal  of  research  in  simulator  visual  displays  to 
identify  those  essential  cues  which  allow  the  pilot  to  control  the 
attitude  of  this  aircraft  and  to  position  it  in  three  dimensional  space 
{ in  accordance  with  the  objectives  and  goals  of  the  moment.  Further,  it 
is  the  objective  of  that  research  to  determine  what  assists  or  Instruc- 
tional aids  may  be  added  to  the  visual  display  to  enhance  the  student’s 
attainment  of  proficiency  in  the  use  of  these  cues.  It  is  the  function 
of  maneuvers  in  the  pilot  training  syllabus  to  practice  the  student  In 
the  accurate  attitude  and  positioning  control  of  his  aircraft.  Certain 
maneuvers  have  specific  and  universal  objectives  such  as  landing  the 
aircraft.  However,  in  the  main,  maneuvers  are  not  practiced  for 
maneuvers'  sake  but  rather  to  practice  the  student  in  control  of  the 
various  dimensions  of  movement  of  the  aircraft  over  which  the  student 
must  have  control, in  order  to  accomplish  specific  objectives  and  pur- 
poses. Therefore,  it  is  suggested  that  the  visual  display  for  the 
beginning  student  must  he  composed  of  that  essential  set  of  features 
which  allows  him  to  exercise  control  about  the  aircraft  axes  and  along 
three  dimensional  space  axes  as  a general  set  of  skills.  He  must  then 
learn  to  extract  from  the  real  world  scene  objects  which  serve  him  in 
carrying  out  bis  control  and  navigational  functions.  He  must  learn 
also  to  discriminate  these  referents  when  they  are  buried  In  visual 
’’noise".  The  term  visual  noise  Is  used  here  to  mean  conditions  In  the 
real  world  environment  which  tend  to  obscure  or  make  difficult  to  dis- 
criminate the  referents  which  the  pilot  wishes  to  use.  Atmospheric 
attenuation,  smoke,  haze  or  any  condition  which  causes  an  object  to  have 
low  definition  constitutes  such  noise.  To  the  extent  that  this  noise 
da grades  the  pilot’s  discrimination  of  changes  between  the  referents 
which  he  Is  using  to  control  his  aircraft  his  precision  of  control  must 
be  degraded. 
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The  analysis  of  the  aye-movement  records  will  provide  relevant  data 
for  Identifying  the  characteristics  Of  those  objects  which  must  be 
Identified  to  attain  the  goals  of  the  pilot  and  to  identify  those 
factors  in  the  environment  which  tend  to  inject  noise  into  the  visual 
scene.  The  objects  which  must  appear  in  the  visual  display  to  allow  him 
to  exercise  attitude  and  position  control  are  not  of  necessity  local 
terrain  features  and  could  be  quite  abstract.  However,  it  will  add 
appreciably  to  the  realism  and,  therefore,  the  acceptance  of  the  display 
If  they  are  recognizable  as  local  features. 

The  identification  and  discrimination  of  changes  among  objects  may 
be  hypothesized  to  be  a function  of  the  level  of  detail  of  the  object 
and  Its  contrast  with  the  background.  These  factors  are  controllable 
as  experimental  variables  in  ASUPT  and  may  be  varied  systematically 
through  the  use  of  the  number  of  lines  used  to  define  an  object  and  the 
shades  of  gray  used  to  provide  contrast  between  objects  or  between  ob- 
jects and  backgroun  ' (figure-ground  contrast). 

In  the  light  of  the  above  discussion  the  first  experiments  dealing 
with  the  content  of  the  visual  display  are  divided  into  the  two  cat- 
egories of  (I)  that  content  necessary  for  control  of  the  attitude  and 
position  of  the  aircraft  in  three  dimensional  space,  and  (2)  those 
relevant  to  the  Identification  and  use  of  features  of  the  environment 
which  allow  the  pilot  to  control  his  vehicle  in  attainment  of  some 
specifiable  goal  or  purpose.  Results  from  the  first  of  these  wl! I be 
appropriate  to  tho  specification  of  displays  for  simple  basic  aircraft 
contro'  trainers.  The  results  from  both  provide  information  for  specifi- 
cation of  more  complex  mission  oriented  trainers.  The  variables  and 
their  levels  for  studying  attitude  and  position  control  are  given  in 
Table  6. t . 

This  expariment  illustrates  the  necessity  for  establishing  the 
discriminable  intervals  of  a variable  before  going  into  the  experiment 
proper,  in  the  choice  of  the  shades  of  gray  for  any  object  it  is 
evident  that  if  shade  of  gray  is  to  be  varied  over  several  levels  those 
levels  must  be  chosen  which  are  discrimlnabiy  different  by  the  subject. 
Should  any  two  levels  chosen  not  be  discriminable  it  follows  that  they 
actually  represent  one  level  of  the  variable  to  him. 

The  discriminable  levels  of  gray  may  be  established  quickly  simply 
by  asking  the  subject  to  fly  the  display  while  observing  the  shades 
of  gray  In  specified  objects.  The  shades  of  gray  in  these  objects  are 
then  varied  up  and  down  the  scale  with  the  subject  reporting  each  time 
he  discriminates  a change  in  shade.  This  simple  psychophysical  procedure 
will  serve  to  establish  those  intervals  minimal iy  perceptible  by  the 
subject  while  he  Is  engaged  in  his  flying  task.  Establishing  these 
Intervals  allows  the  selection  of  levels  of  the  variable  demonstrably 
different  to  the  subject  for  use  in  transfer  of  training  studies. 
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Table  6.1.  Visual  display  expe,  i manta  I conditions  - attitude  and 
position  control  Study  3. 


m 


$ 


i 


m 


k 


The  second  major  category  of  study  is  the  determination  of  those 
features  of  the  environment  which  allow  the  pilot  to  change  his  position 
or  behave  relative  to  those  features  in  some  purposeful  way.  The  two 
major  tasks  in  UPT  in  this  category  are  (I)  traffic  pattern,  approach 
and  landing,  and  (2)  formation  flying.  Of  these  two  training  problems 
that  of  traffic  pattern  and  approach  to  landing  has  been  researched 
to  a greater  extent  than  that  of  formation  flying.  However,  recent 
experiments  at  HRL/F'i  using  the  Formation  Flight  Trainer  (FFT)  has 
provided  directly  applicable  data  to  the  specification  of  trainers  for 
training  in  this  maneuver. 


The  ASUPT  visual  display  capability  allows  the  study  of  the  re- 
levant referents  to  be  presented  for  the  touchdown  phase  of  the  landing 
and  how  they  may  be  encoded  in  the  display.  Satisfactory  study  of  this 
area  has  not  been  possible  up  to  this  time.  The  ASUPT  capability  makes 
it  possible. 


~ The  variables  and  suggestions  for  their  encoding  for  the  approach 
and  landing  study  are  given  in  Table  6.2.  It  Is  proposed  that  the 
essential  features  listed  under  Conditions  I and  2 in  both  Tables  2 and 
3 may  ba  better  defined  after  analysis  of  the  eye-movement  records  and 
the  full  capability  of  the  ASUPT  for  providing  display  features  has 
been  completely  checked  out. 


The  procedure  for  establishing  the  ASUPT  criterion  system  in 
Study  I has  been  discussed  in  Section  4.0, Methodological  Considerations. 
Study  2 will  employ  the  classical  transfer  of  training  paradigm  dis- 
cussed by  Gagne,  Foster  and  Crowley  (1948). 


Studies  3 and  4 are  proposed  as  tests  of  the  performance  equiva- 
lence approach  as  well  as  to  assess  the  transfer  to  the  aircraft  of 
less  complete  visual  systems.  The  parameters  of  number,  position  and 
stylization  of  objects  will  be  varied  so  that  (Da  condition  not  equi- 
valent to  the  criterion  ASUPT  Is  obtained  and  (2)  a condition  equivalent 
to  the  criterion  ASUPT  system  Is  obtained.  This,  is  to  be  done  in  the 
manner  described  in  Section  4.0  using  the  performance  of  experienced 
pilots.  Classical  transfer  of  training  experiments  are  then  proposed 
to  be  conducted  using  Conditions  I and  2 of  these  studios  to  determine 
whether  the  hypothesis  that  equivalent  systems  bring  about  the  same 
amount  of  transfer  while  non-equivalent  systems  do  not.  The  amount  of 
transfer  obtained  earlier  in  Study  2. 


6.2. 


The  general  design  format  for  Studies  3 and  4 is  shown  in  Figure 
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6. 1.2. I Genera! 


The  problem  of  determining  the  necessary  degrees  of  freedom  of 
cockpit  motion  In  training  simulators  was  sixth  on  the  priority  list 
; of  training  problems.  (See  Appendix  0)  Manipulation  of  this  variable 

| in  experimental  investigations  Is  particularly  easy  using  the  ASUPT. 

1 This  device  also  has  additional  areas  of  interest  to  trainer  motion 

l dosign  - that  of  gravity  alignment  and  the  G-seat,  Gravity  alignment 

i cuaing  is  designed  to  come  from  tilting  the  platform  such  that  normal 

| gravity  is  used  to  substitute  for  the  acceleration  or  declaration  of 

j.  the  aircraft  particularly  during  take-off  or  landing  G-seat  cueing  is 

j of  two  forms  - those  which  come  from  movement  of  the  elements  of  the 

J seat  to  provide  movement  cues  and  those  which  provide  sustained 

pressures  to  the  student’s  body.  See  Appendix  B for  more  complete 
j definitions. 

; As  pointed  out  by  Smith  (1972)  it  is  manifestly  not  possible  or 

sensible  to  attempt  to  investigate  ail  of  the  possible  combinations  and 
permutations  of  the  six  degrees  of  freedom  of  motion,  gravity  alignment 
and  the  ways  in  which  the  G-seat  may  be  used.  A sorting  out  on  the 
basis  of  the  characteristics  and  dynamics  of  the  physical  system  being 
simulated  and  of  the  human  sensory  system  was  felt  necessary  and  has  been 
done.  Further  screening  of  these  variables  is  proposed  to  te  carried 
out  in  pretest  investigations  as  indicated  in  Figure  5.1* 

The  combinations  of  conditions  selected  for  recommended  study  and 
listed  in  Table  6.3  may  be  used  either  in  classical  transfer  experiments 
- or  they  may  be  approached  through  the  performance  equivalence  method. 

in  the  discussion  to  follow  the  performance  equivalence  method  is  used 
although  the  fact  of  the  previous  statement  must  be  kept  In  mind. 

6. 1.2.2  Experimental  Studies 

The  specific  experimental  question  to  be  addressed  by  these  Investi- 
gations is  ’’under  what  method  of  introducing  motion  cues  and  across  what 
conditions  of  flight  are  the  control  performances  of  experienced  pilots 
equivalent?”  The  approach  taken  supposes  that  if  two  systems  are 
measurably  different  by  some  quantitative  metric  but  their  operation 
results  in  no  measurabie  difference  in  the  performance  of  experienced 
pilots  thon  the  physical  differences  are  behavioral ly  equivalent  ai.d  are 
not  different  for  purposes  of  a training  simulator.  This  is  the  method- 
ology discussed  in  Section  4.0. 

The  independent  variable  of  prime  Interest  is  that  of  the  motion  cue 
condition.  What  is  sought  is  a determination  of  whether  pilot  perform- 
ance varies  as  a function  of  motion  cue  conditions  across  different 
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maneuvers  and  for  different  levels  of  external  disturbances.  The 
motion  cues  to  be  investigated  fall  under  the  three  broad  categories 
just  discussed:(a)  platform  motion,  (b)  gravity  alignment  and  (c) 

G-seat.  Particular  combinations  of  sub-categories  of  these  major  areas 
have  been  selected  to  obtain  data  about  ten  specific  questions.  These 
conditions  are  listed  in  Table  6.3.  The  specific  experimental  questions 
are  given  In  Table  6.4. 

-•Table  6.4  shows  the  experimental  conditions  relevant  to  providing 
information  about  particular  experimental  questions.  The  experimental 
procedure  is  one  in  which  the  twenty-two  conditions  may  be  run  in  three 
experimental  sessions  and  provide  information  relevant  to  the  ten 
experimental  questions  posed.  The  experimental  questions  have  been 
selected  such  that  successive  data  runs  can  be  made  with  experimental 
conditions  on  subsequent  runs  being  dependent  upon  the  findings  of  earlier 
runs.  A "sorting  out"  procedure  is  used  in  which  the  various  combin- 
ations r f conditions  are  examined  to  determine  whether  thoy  effect  any 
differences  In  pilot  performance  and  what  the  relative  contribution  of 
the  conditions  are  to  performance  variability*  The  effects  found  under 
one  set  of  conditions  are  used  to  guide  the  selection  of  the  experimental 
conditions  to  be 'used  in  the  next  run.  In  order  to  accomplish  this  it 
is  proposed  that  the  incremental  design  outlined  by  Demaree  in  "A 
Recommended  Design  for  Experimental  Studies  Using  ASUPT"  be  used.  This 
paper  Is  given  in  Appendix  H. 


Under  the  proposed  experimental  procedure  as  outlined  in  Table  6.4, 
the  first  experimental  question  would  incorporate  experimental  Conditions 
I,  10,  II,  15,  16,  and  17.  Results  from  performance  under  these  conditions 
are  intended  to  show  whether  or  not  the  major  conditions  of  simulation 
of  (a)  no  motion,  (b)  six  degree  of  freedom  platform-  (c)  six  degree 
of  freedom  platform  with  gravity  alignment,  (d)  six  degree  of  freedom 
platform  with  full  G-seat,  (e)  six  degree  of  freedom  plarform  with  full 
G-seat  and  gravity  alignment , and  < f ) full  G-seat  only,  differentially 
affect  performance  and.  If  so,  what  the  relative  effects  are. 

However,  as  a test  of  the  performance  equivalence  approach  it  is 
proposed  that  Conditions  1 (no-motion),  9 (full  motion),  and  that 
condition  found  to  be  equivalent  to  full  motion  be  used  In  the  paradigm 
shown  in  Figure  6,3. 

In  this  paradigm  equivalence  and  non-equivalence  is  established  by 
the  methods  outlined  in  Section  4.0.  In  Figure  6.3  the  condlTion  of 
no-motion  Is  assumed  to  he  non-equivalent  to  the  ASUPT  criterion  system 
in  terms  of  performance,  it  is  also  assumed  that  a condition  of  less 
than  the  full  criterion  ASUPT  system  may  be  ostablished. 

If  Experimental  Conditions  i and  10  differentially  affect  performance 
Session  Two  will  be  concerned  with  obtaining  data  on  Experimental 
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Table  6,3 


Experimental  Conditions  Selected  for  Test  in  Determining 
Platform  Moiion  and  G-Seat  Operation 


Experimental  Condition 

Condition  Description* 


i « 
2, 

3. 

4. 

5. 

6. 

7, 

8, 
9. 

10. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 
21. 
22. 


No  Motion 
P,R  (Platform) 

P,  R,  Y (Platform) 

P,  R,  Y,  H (Platform) 

P,  R.  H (Platform) 

P,  R,  H,  L (Platform) 

P,  R,  H,  f i A (Platform) 

P,  R,  H,  L,  F 4 A (Platform) 

P,  R,  Y,  H,  L (Platform) 

6 Degree  of  Freedom  Pi  at form 
6 Degree  of  Freedom  Platform  with  Gravity 
A I ignment 

P,  R (Platform)  with  Gravity  Alignment 
P , R,  H (Platform)  with  Gravity  Alignment 
P,  R,  Y,  H,  L (Platform)  with  Gravity 
Alignment 

6 Degree  of  Freedom  Platform  with  Full 
G-seat 

6 Degree  of  Freedom  Platform  with  Ful I 

G-seat  and  Gravity  Alignment 

Ful I G-seat  only 

P,  R (G-seat),  H (Platform) 

P,  R (Platform(,  H (C-seat) 

P,  R (G-seat) 

P,  R,  F & A (Platform) 

H only 


*Legend: 

P - Pitch 
R - Roll 
Y - Yaw 
H - Heave 
L - Lateral 
F&A  - Fore-and-aft 


45 


e.nfrrf  evi-. 


E 

9 t 

4-  u 
W — 

- >-  o 
O V) 

+-  o 

Irt  4- 

T3 

® -C  T> 
Cf  « 


WO  c 
4-  — 9 
C l-  L. 

9 9 4-4- 
"0  4-  •+- 

3 ~ -a  a 
+■  u.  c t. 
to  y n y 


> w 

l 0)4- 

. E C O 

9 

• 4~  IA  — 

■ w 3 a 
> 

I W TJ  TO 
i ® m 

S-xz  0 

«»  C 

it-  ® 

« X>  1_ 
4-  9 o 
~ 4-  CL 
r <-  W X 
U ffi  ® 


E 

£ i 

w — 

^ >>  9 
O W 
4-  , 0 

9 4- 
O — 

® £ O 
C 4-  g 

9 C L. 
t-  O 9 
4-  4- 

SW 

c 

4-  •—  9 
C i-  u 
9 9 4-4- 
•0  4-  '4- 

3 — O 9 
-t-  t-  C «- 
</)  O 9 O 


Conditions  2,  3,  4,  5,  6,  7,  8,  9,  and  21.  The  results  from  these 
conditions  are  intended  to  determine  what  combination  of  degrees  of 
freedom  of  motion  of  the  platform  are  equivalent. 

Experimental  question  No.  3 is  answered  by  comparison  of  Conditions 
10  and  II  with  experimental  run  I.  This  question  concerns  the  contri- 
bution that  the  gravity  alignment  cue  makes  to  the  maximum  degree  of 
freedom  platform. 

In  Experimental  Session  3 the  six  remaining  experimental  conditions 
of  12,  13,  14,  18,  19,  and  20  are  added.  Comparison  of  Experimental 
Condition  13  with  Condition  5 is  intended  to  answer  the  question  with 
respect  to  a contribution  of  gravity  alignment  given  a nominal  platform. 
What  constitutes  a nominal  platform  has  been  assumed  to  be  that  re- 
presented by  Condition  5,  i.e.,  one  with  pitch,  roll  and  heave.  How- 
ever, the  results  of  the  analysis  of  Experimenlal  question  2 may  change 
what  Is  considered  to  be  a nominal  platform  and,  therefore,  change  these 
conditions  somewhat. 

Comparison  of  Conditions  15  and  16  is  designed  to  give  information 
about  the  contribution  of  gravity  alignment  when  the  maximum  platform 
is  used  with  rhe  G-seat. 

Addition  of  Condition  12  allows  the  comparison  between  Condition  2 
and  12  and  gives  Information  about  the  contribution  of  gravity  alignment 
with  the  minimum  platform,  i.e.,  one  which  provides  only  pitch  and  roll 
sttmul S . 

The  addition  of  Condition  14  allows  for  a comparison  of  that 
condition  with  Conditions  9 and  10  and  gives  information  about  the 
longitudinal  acceleration  cue  contribution  when  a maximum  platform  is 
used,  i.e.,  whether  the  aravity  alignment  or  fore-and-aft  translation 
is  the  better  clue  or  wnether  it  has  any  effect  at  ail  upon  performance. 

In  carrying  out  the  experimental  runs  it  is  proposed  that  the  ex- 
perimental conditions  be  carried  out  under  Instrument  flight  conditions 
using  the  maneuvers  of  straight  and  level  flight,  30°  bank  turns  and 
unusual  attitudes.  It  In  also  proposed  that  the  fundamental  nature  of 
the  control  task  required  of  the  pilot  in  performing  these  tasks  is  a 
function  of  the  external  forcing  function  or  turbulence  acting  upon 
the  system.  His  control  behavior  will  be  affected  by  the  nature  of 
this  disturbing  function  and  it  is,  therefore,  proposed  that  three 
leve-ls  of  disturbance  be  imposed  upon  each  of  the  basic  flight  tasks  in 
order  to  determine  the  equivalence  of  various  physical  systems  across 
representative  Tasks  and  disturbances.  The  ievel  of  disturbance  for 
each  condition  is  to  be  determined  through  pre-testing  of  the 
turbulence  generation  system  of  the  ASUPT. 
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6.1.3  Motion-Vision  Interaction  Studies 


6,1,3. I General 

The  results  of  certain  experiments  conducted  In  the  visual  and 
motion  areas  are  proposed  to  be  used  In  experiments  designed  to  provide 
Information  about  the  Interactive  effects  of  these  two  major  variables. 
These  experiments  are  proposed  to  be  conducted  using  the  two  major 
areas  of  visual  cueing  discussed  In  Section  6.;.!.  These  are  aircraft 
attitude  control  and  aircraft  position  confrol.  Within  these  two  areas 
experiments  are  designed  to  determine  whether  differential  Interactive 
effects  occur,  — 


6. 1.3.2  Experimental  Studies 

As  mentioned,  the  Interactive  effects  of  motion  and  vision  are 
proposed  to  be  Investigated  In  two  rather  different  areas  cf  pllc+  tasks. 

First,  are  those  conditions  In  which  the  pilot’s  task  Is  control  of  the 
attitude  of  his  aircraft.  This  control  Is  primary  and  antecedent  to 
control  of  the  position  of  the  aircraft  In  three  dimensional  space. 

It  Is  hypothesized  that  the  Interactive  effect  Is  Influenced  by 
numerous  variables  In  the  total  task  situation.  Primary  among  these 
variables  are  level  of  precision  and  control  required  and  the  nature  of 
the  external  forcing  functions  (ordinarily  termed  turbulence)  Imposed 
upon  the  system. 

The  model  of  control  behavior  adopted  In  considering  aircraft 
attitude  control  Is  that  of  the  Effective  Time  Constant  (t0)  of  the 
system.  (Matheny  & Norman,  1968)  This  model  is  used  as  a basis  for  under- 
standing the  Interactive  effects  as  well  as  to  form  predictive  hypotheses 
to  be  tested.  In  brief,  the  model  assumes  that  the  precision  of  closed-loop 
error-nul I Ing  behavior  is  a function  of  the  Immediacy  of  feedback  to  the 
controlling  operator,  e.g,,  the  pilot.  The  time  taken  for  feedback  to  occur 
(*as  been  termed  the  Effective  Time  Constant  of  the  man/machlne  system.  The 
value  of  t depends  upon  the  values  of  given  properties  of  the  machine  dynamics 
and  of  the  human  controller.  The  properties  of  fhe  human  controller  of 
Importance  are  tho  modalities  through  which  the  Information  Is  received 
and  the  threshold  level  of  those  modalities. 


The  relevance  of  this  model  to  the  motlon/vlslon  interaction  problem 
Is  that  certain  Information  about  the  status  of  the  aircraft,  particu- 
larly changes  In  attitude  of  the  aircraft,  are  transmitted  more  Immedi- 
ately as  feedback  to  the  operator  through  the  motion  senses  than  the 
visual,  it  Is  a tenet  of  the  model  that  the  motion  senses  are  cued  by 
rates  of  onset  of  acceleration  which  allow  for  the  Initiation  of  re- 
sponse much  in  advance  of  that  which  would  be  triggered  by  the  resulting 
positional  change  of  the  stimulus  detected  by  the  visual  sense.  It 
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follows  from  the  model  that  Increasing  the  gain  on  a given  display  to 
a particular  sense  will  Increase  the  Immediacy  of  the  feedback  to  the 
operator.  Thus,  Increasing  the  gain  of  the  visual  display  wtil  result 
In  an  Increase  In  the  precision  of  control.  Likewise,  visual  displays 
which  provide  the  operator  with  rate,  acceleration  or  onset  of  accel- 
eration Information  allow  him  to  receive  more  Immediate  Information  as 
to  the  status  of  the  system  relative  to  the  desired  state  and  to  control 
It  more  precisely. 


From  tl.e  Effective  Time  Constant  model  the  prediction  can  be  made 
that  In  systems  such  as  the  T-37  aircraft,  performance  of  the  precise 
attitude  control  task  will  be  enhanced  by  addition  of  the  proper  motion 
cues  while  other  tasks  such  as.'*  positional  control  In  which  the  visual 
feedback  Is  timely  and  adequate  will  not  benefit  greatly  from  the 
addition  of  motion  cues.  It  can  be  further  predicted  that  the  visual 
display  which  Inccvorates  the  higher  gain  .(e.g,,  large  contact  display) 
will  result  In  a higher  precision  of  control,  it  can  also  be  hypothesized 
that  training  a student  to  a given  level  of  proficiency  without  those  motion 
cues  which  will  be  present  In  the  transfer  tasks  constitutes  over  training 
him 'and  he  will  do  very  well,  l.e.,  transfer  will  be  high,  when  he  Is 
transferred  Into  the  task  In  which  the  total  set  of  stimulus  cues  are  avail- 
able. 


As  mentioned  earlier  the  first  Interaction  experiments  are  designed 
to  examine  the  situation  In  which  Interaction  effect  is  likely  to  be 
greatest  as  predicted  by  the  effective  time  constant  mode!.  This  area 
Is  In  that  of  close  precision  attitude  control.  The  experimental  conditions 
suggested  for  this  experiment  are  given  In  Table  6,5, 

It  Is  proposed  that' the  performance  equivalence  paradigm  be  used 
for  this  experiment.  Experienced  pilots  will  fly  each  of  the  conditions 
and  tasks  to  determine  whether  or  not  their  output  control  behavior  Is 
different  under  the  various  conditions  given  standard  system  output- 
performance  requirements.  For  those  conditions  found  to  be  equivalent 
and  those  found  not  to  be.  It  is  proposed  that  groups  of  the  si udents 
be  trained  under  each  condition  and  subsequently  transferred  to  Jtie  air- 
craft as  a further  test  of  the  performance  equivalence  approach. 
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Table  6.5 

Experimental  Conditions  for  Initial  Motion-Vision 
Interaction  Experiments 


Tasks:  Approach  to  landing. 

Straight  and  level. 

Variables:  Motion  - no  motion  and  full  6 degree. 

Turbulence  - high  and  low. 

Precision  of  control  - high  and  low. 

Subjects:  Experienced  :4lots  for  establishing 

performance  equivalence.  Trainees  for 
training  on  selected  conditions  ana 
transfer  to  T-37  aircraft. 


i 

i 
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6.2  Training  Methodology  Research 

As  Indicated  In  the  Introductory  section,  this  area  of  research  Is 
proposed  to  have  high  potential  for  producing  results  which  will  greatly 
Increase  training  effectiveness  and  has  Immediate  application.  The 
adoption  of  ASUPT  as  a criterion  system  for  investigating  efficient 
training  methods  Is  the  basis  for  this  proposal.  This  Is  particularly 
true  of  research  In  two  areas  judged  to  be  Important  by  the  training 
problems  panel;  namely,  cognitive  pre-training  and  feedback.  These  two 
areas  will  be  discussed  first  followed  by  recommendations  regarding  the 
areas  of  sequencing  of  training  tasks,  contextual  training  and  Individual- 
ized Instruction. 

6.2.1  Cognitive  pre-training 

Cognitive  pne-t raining  refers  tr  th?  complete  understanding  of  the 
cognitive  and  action  aspects  required  for  completion  of  a task  before  that 
task  is  attempted  In  the  simulator  or  aircraft.  In  practice  It  may  be 
achieved  In  several  different  ways. 

For  the  perceptual  and  cognitive  aspects  of  tasks,  such  as  learning 
to  scan,  read  and  Interpret  the  Instruments  for  Instrument  flight  or 
foaming  procedural  sequences,  a numbe;  of  Instructional  aids  such  as 
photos,  work  books,  or  audio-visual  devices  may  be  hypothesized  to  provide 
effective  training  in  these  task  aspects.  Cognitive  understanding  or 
mental  pre-practlcc  may  be  highly  effective  pre-training  for  continuous 
control  tasks  through  use  of  simple  or  even  no  equipment  at  all. 

It  Is  proposed  that  the  evidence  Is  sufficient  to  support  Institution 
of  certain  forms  of  pre-cognitive  training  directly  Into  the  training 
program.  These  Involve  those  techniques  demonstrated  to  be  efficient  by 
Flexman,  et  a|.  (1950)  and  Flexman,  et  al,  (1954).  In  general  these 
techniques  require  the  student  to  become  proficient  In  his  ability  to 
verbalize  the  pertinent  cues  and  responses  necessary  to  meet  the  require- 
ments of  the  task.  In  this  verbalization  It  is  Important  that  the 
instructor  determine  whether  the  student  is  merely  ’’parroting"  the  words 
of  the  instructor  or  whether  he  completely  and  thoroughly  understands  the 
perceptions  and  responses  that  are  required  for  successfully  accomplishing 
the  maneuver.  With  the  advent  of  Increasing  initial  and  maintenance 
costs  of  simulators,  the  Investigation  of  the  amount  of  time  which  might 
be  saved  in  reaching  criteria  In  the  ASUPT  simulator  for  certain  maneuvers 
through  the  use  of  other  pre-cognitive  training  methods  Is  highly 
recommended. 

The  Human  Resources  Laboratory/Flying  Training  Division  has  as  a 
part  of  its  research  capability  an  audio-visual  Instrument  training 
device  (AVIT)  with  a program  suitable  for  pre-cognitive  training  In 
scanning,  reeding  and  interpreting  the  basic  instruments  for  basic 
instrument  flight.  The  contribution  of  this  device  to  reduction  of 
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simulator  training  time  for  learning  the  basic  flight  maneuvers  capable  of 
being  taught  by  the  device  should  be  Investigated.  The  device  Is  con- 
figured to  teach  a student  to  scan,  read  and  Interpret  his  Instruments  for 
straight  and  level  flight,  30°  bank  turns  and  unusual  attitudes.  A 
functional  description  of  the  device  is  given  In  Appendix  G. 

It  Is  proposed  that  an  experimental  group  be  taught  to  criterion 
level  In  the  AVIT  device  and  compared  to  a control  group  taught  only  In 
the  simulator  in  terms  of  time  to  reach  proficiency  In  the  simulator.  It 
Is  proposed  that  both  the  experimental  and  the  control  groups  then  be 
taught  all  other  maneuvers  in  the  basic  Instrument  flight  curriculum  to 
determine  the  degree  of  generalization  from  the  three  maneuvers  taught  In 
the  AVIT  to  other  maneuvers  performed  In  the  simulator. 

It  Is  proposed  also  that  the  capaLMlty  for  a device  such  as  the 
AVIT  be  Investigated  for  its  use  in  teaching  the  perceptions  required  In 
contact  maneuvers  such  as  the  approach  to  landing.  This  hypothesizes 
that  the  major  portion  of  the  learning  task  of  the  pilot  trainee  Is  the 
learning  of  the  perceptual  and  cognitive  aspects  of  the  task  as  opposed  to 
the  motor  responses  he  Is  required  to  make.  It  Is  hypothesized  that  once 
the  pilot  has  learned  to  perceive  the  desired  stimulus  relationships,  to 
recognize  departures  from  these  desired  relationships  and  to  know  In  which 
direction  he  should  Initiate  control  movements  In  order  to  correct  them 
his  training  time  In  the  actual  performance  of  the  continuous  time  varying 
task  will  be  greatly  reduced. 

In  brief,  it  Is  recommended  that  the  use  of  a programmed  branching 
audio-visual  Instrument  trainer  be  investigated  for  teaching  the  perceptual 
and  cognitive  side  of  contact  tasks  since  these  tasks  are  among  the  most 
demanding  of  the  trainee  and  require  some  of  the  most  complex  equipment 
for  simulation. 

6.2.2  Feedback 

In  general  Zeitgeist  has  been  that  feedback  or  knowledge  of  results 
facilitates  training,  (see  Smode,  T958)  While  this  Is  true  In  the 
general  case  there  are  cautions  which  should  be  observed  when  applying  the 
feedback  principle  to  the  particular  case  or  when  carrying  out  research 
with  particular  tasks.  For  example,  Briggs  (1962)  points  to  the  Inter- 
active effect  of  type  of  augmented  feedback  (positive  or  negative),  com- 
plexity of  the  feedback  criteria  and  level  of  training.  He  draws  attention 
also  to  the  Importance  of  the  feedback  withdrawal  schedule.  Ward  and 
Senders  (1966)  suggest  that  adding  an  element  to  the  task  such  as  a 
feedback  method  has  a degree  of  workload  associated  with  It  and  may  Interfere 
with  carrying  out  the  primary  tasks  due  to  the  time  sharing  requirements. 

The  work  of  Feuerzelg  (1971)  In  providing  Instructional  monitoring 
Is  believed  to  be  a method  of  feedback  which  should  prove  highly 
beneficial  In  Undergraduate  Pilot  Training  but  must  be  Investigated  as 
to  the  Interference  effects  and  the  feedback  withdrawal  schedule. 
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Feuerzeig  provided  the  trainee  with  computer  generated  diagnostic  infor- 
mation, instruct’/onai  suggestions  and  a two-dimensional  dynamic  display 
of  the  progress  through  a task  such  as  a holding  pattern.  His  results 
suggest  that  sroh  an  approach  to  feedback  to  the  UPT  trainee,  particularly 
for  instrument  flight  training,  should  be  investigated. 
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6,2.3  Sequencing  of  Training  Tasks 

While  sequencing  of  training  tasks  was  listed  as  being  one  of  tne 
top  priority  items  by  the  Training  Problem  Panel,  it  Is  suggested  that 
the  solution  to  this  problem  is  more  properly  carried  out  using  the  ISO 
task  analysis  approach  suggested  in  Manual  50-2.  How  best  to  teach  each 
of  the  separate  tasks  may  be  approached  through  research  in  training 
methodology  such  as  those  of  cognitive  pre-training  and  feedback.  It  is, 
recommended  that  the  proper  sequencing  of  these  tasks,  however,  may  more 
properly  be  done  through  synthesizing  the  tasks  Into  an  overall  curriculum 
by  the  use  of  ISD  procedures.  The  comparative  test  of  two  optimized 
sequences  may  be  carried  out  as  a final  test  using  the  ASUPT  as  the 
criterion  device  within  which  all  training  is  given  and  final  training  Is 
tested.  This  final  comparative  experiment  should  be  undertaken  after 
thorough  analyses  of  the  tasks  required  of  the  pilot  and  should  properly 
include  a synthesis  of  the  tasks  of  the  total  curriculum  based  upon  the 
analysis  carried  out  by  Meyer  et  al,  (1974). 
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6.2.4  Contextual  Training 

The  arguments  for  investigation  of  contextual  training  are  Identical 
to  those  just  presented  for  sequencing  of  the  training  tasks.  The  context 
within  which  certain  tasks  are  to  be  trained  should  be  established  and 
synthesized  by  task  analysis  prior  to  any  experimental  investigations. 
Comparative  studies  of  two  or  three  of  the  candidates  for  "contextual" 

1 raining  should  then  be  investigated  in  ASUPT  using  it  as  both  the  training 
and  the  criterion  device. 

6.2.5  Individualized  Instruction 
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Individualized  instruction  is  such  a broad  and  diversified  area  that 
it  is  not  possible  to  describe  research  relevant  to  it  in  this  report. 

The  research  outlined  with  respect  to  training  methodologies  such  as 
feedback  and  cognitive  training  will  provide  Information  as  to 
instructional  strategies  which  may  go  into  an  individualized  instructional 
system.  It  is  recommended  thai  the  individual  characteristics  such  3s 
biographical  data  and  subject  source,  i.e..  Air  Force  Academy,  ROTA  and 
so  forth  be  maintained  on  all  trainees  used  as  subject  in  the  experiment. 
The  predictive  validity  of  research  findings  will  rest  in  large  part  on 
information  about  present  and  future  trainee  populations.  Thus,  a data 
base  may  be  developed  for  making  management  decisions  about  Individual- 
ized instructional  programs  which  may  be  automated  to  assign  instructional 
strategies,  incentives  and  guide  the  trainees’  progress  as  a function  of 
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hfs  ra+e  of  skill  attainment,  biographical  background  and  other  In- 
dividual characteristics  upon  which  he  may  have  been  selected.  The  re- 
search being  carried  out  by  Human  Resources  Laboratory,  Technical  Training 
Division,  with  respect  to  individualized  training  will  provide  useful  data 
for  management  decisions  about  individualized  Instruction  In  flying 
training  and  should  form  a part  of  the  data  base. 


I 
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APPENDIX  A 


Letter  of  Invitation  anu  Instructions  to 
Research  Problems  Panel 


Life  Sciences,  Inc,,  under  Contract  F4 1609-73-00038  with  the 
United  States  Air  Force,  Is  carrying  out  an  Investigation  th6  purpose  of 
which  Is  to  produce  a Handbook  of  experiments  dealing  with  the  design 
and  utilization  of  training  simulators.  This  document  will  outline  the 
more  Important  experiments  necessary  of  being  carried  out  In  order  to 
answer  critical  questions  regarding  simulator  hardware  design  and  methods 
of  simulator  utilization.  The  research  to  be  recommended  In  this  Hand- 
book Is  to  be  carried  out  using  the  Advanced  Simulator  for  Undergraduate 
Pilot  Training  (ASUPT)  or  other  research  facilities  available  to  the  Air 
Force  Human  Research  Laboratory/F! Ight  Training  at  Williams  Air  Force 
Base. 


It  Is  expected  that  the  problems  to  be  attacked  will  be  those  im- 
portant to  a wide  range  of  training  situations  In  which  the  trainee 
acquires  cognitive  and  psychomotor  skills  In  learning  to  control  a man- 
machine  system.  However,  our  emphasis  Is  upon  the  research  through  use 
of  the  ground-based  trainer  and  training  technology  In  beginning  pilot 
training. 

In  order  to  make  the  Handbook  of  most  value  to  Air  Force  research 
and  to  ensure  the  most  generality  across  a wide  range  of  beginning 
pilot  training  programs,  we  are  c-ttemptlng  to  involve  as  many  Individuals 
with  experience  and  knowledge  In  the  problems  of  beginning  training  as 
possible.  This  involvement  takes  the  form  of  asking  the  Individuals 
to  present  their  views  as  to  the  most  critical  training  problems  to  be 
attacked  using  a facility  such  as  ASUPT  and  the  oTher  HftL/FT  devices. 
Therefore,  we  are  attempting  to  obtain  the  considered  judgments  of  a 
group  of  experts  in  the  field  as  to  tht,  important  training  problems. 

After  these  have  been  assembled  they  will  be  listed  and  submitted  back 
to  the  group  for  priority  judgments  or  order  in  which  they  should  be  taken 
up  in  a research  program. 

I would  appreciate  very  much  your  becoming  a member  of  this  group. 
Your  participation  as  a member  would  be  most  valuable. 

I should  point  out  that  other  equipment  available  for  research  on 
training  problems  consists  of  a T-4G  simulator  which  is  a modification  of 
the  MF-l  built  by  Link.  Its  main  features  are  a mostlon  base  and  a 
visual  system.  The  motion  system  provides  three  degrees  of  motion  In 
pitch,  bank  and  heave.  The  visual  display  provides  a visual  image  of 
44°  by  28°  In  full  color  through  use  of  color  film  and  35  mm  slides. 

This  visual  system  is  known  as  the  Electronic  Perspective  Transformation 
(EPT)  system  and  presents  an  approach,  landing  and  take-off  sequence 
filmed  at  William^  Air  Force  Base. 
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Other  equipment  Includes  three  T-40  trainers  which  were  assigned 
to  train  flight  personnel  In  the  typical  twin  engine  Jet  aircraft.  This 
trainer  consists  of  a cockpit  mounted  on  a two  degree  of  freedom  motion 
system  and  an  Instructor’s  station.  The  motion  system  provides  motion 
In  the  pitch  and  roil  axes,  Howaver>  the  pitch  motion  also  provides 
vertical  translation  at  the  pilot’s  center  of  gravity  to  simulate  heave 
motion.  The  trainer  may  be  considered  as  an  Instrument  flight  trainer 
and  useful  in  research  on  the  problems  of  Instrument  flight,  engine 
systems,  navigation,  radio,  communications  3nd  so  forth  for  this  type 
of  aircraft. 

A third  trainer  is  termed  the  simplified  Formation  Flight  Trainer 
(FFT).  This  device  is  configured  to  train  the  student  In  formation  flying 
of  the  T-38  aircraft.  The  student  "flys"  a television  camera  which  views 
a,  mode!  of  a T-38  training  aircraft.  The  image  of  this  mode!  is  pro- 
jected cn  a wide  angle  screen  at  the  student’s  station.  All  basic  formation 
tasks  from  join  up  to  position  keeping  and  In  cross-under  can  be 
accomplished  in  the  trainer. 

I hope  that  you  will  be  able  to  help  as  a member  of  our  oanel  of 
experts  and  can  send  me  your  thoughts  on  what  you  regard  to  be  the 
critical  research  Issue  in  simulator  design  and  utilization  at  an  early 
date. 

Best  regards, 


W,  G,  Matheny 

President  & Technical  Director 
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Training  Problems  List 


Visual  Simulation  Requirements 


The  visual  system  ir.  aircraft  simulators  presents  the  real  world 
visual  scene  outside  the  aircraft  to  the  trainee.  Major  problems  in  this 
area  are; 

o Field  of  View  - The  solid  visual  angle  to  be  covered  by  tho 
display.  Measured  from  a forward  I ine-of- regard,  the  vertical 
and  hot  izonta!  visual  angles  to  be  covered  by  the  display. 

o Content  of  the  Oisplay  - Objcc/.s,  points,  lines  and  so  forth 
in  the  display  as  sources  of  information  for  the  trainee  in 
carrying  out  his  aircraft  control  tasks. 

o Quality  of  the  Display  - Aspects  of  the  disDiay  such  as 

brightness,  resolution,  sharpness,  contrast,  and  distortion 
or  aberrations  of  the  displayed  image  which  have  to  do  with 
its  legibility. 

o Color  - The  chromatic! tv  of  the  display,  i.e,,  whether  the 
visual  display  is  achromatic  or  contains  an  approximation  to 
real  world  color. 

o Depth  of  Field  - The  image  being  displayed  to  the  trainee  is 
collimated  and  thus  appear  at  virtua*  infinity  or  appear  as 
a flat  plane  projection. 

Motion  Cue  Simulation 


This  area  of  research  may  be  suLdiv’aed  into  what  may  be  termed  the 
onset  or  true  motion  cues  and  their  washout  andvthosc  cuej  which  are 
sustained  by  means  of  pressures  cn  the  trainee’s  body.  The  latter  may  be 
implemented  by  Drieumatical ly  driven  elements  in  the  simulator  seat  (G-Seat) 
and  through  the  gravitational  alingment  cue  provided  by  tilting  the  motion 
platform.  Mofion  cues  may  be  provided  by  the  movement  of  the  simulator 
motion  platform  and  possibly  the  ac+uation  of  the  pneumatical ly  driven 
elements  in  the  G-Sea+.  The  individual  research  items  have  b98n  broken 
out  as  follows: 

o Contribution  of  the  individual  or  combined  degrees  of  freedom 
of  motion  of  the  platform  tc  training  of  the  undergraduate 
pilot  trainee. 


o Contribution  of  the  Individual  or  combined  motions  of  the 
degrees  of  freedom  of  motion  of  the  platform  to  the 
proficiency  measurement  of  the  trainee,  e.g.,  periodic  or 
final  checks, 

o Contribution  of  the  gravity  alignment  cue  to  training.  This 
cue  comes  from  tilting  the  platform  such  that  norma!  gravity 
is  used  as  a substitute  for  the  acceleration  when  the  aircraft 
is  accelerating  or  decelerating,  e.g.„  during  landings, 

o Determination  of  the  degree  to  which  the  seat  pan  and  belt 
pneumatically  driven  elements  (G-Seat)  may  provide  the 
motion  cues  available  1o  the  trainee  during  various  training 
tasks  and  maneuvers. 

o Determination  of  the  contribution  to  training  of  the  sustained 
pressures  provided  by  the  pneumatically  driven  element  of 
the  student's  seat. 

o Determination  of  the  optimum  frequency  response  In  pitch, 

roll  and  yaw  of  the  platform  for  tialnlng  cost  effectiveness, 
l.e.,  what  fidelity  of  motion  Is  optimum  for  training. 

o Determination  of  the  optimum  program  for  introduction  and 
washout  of  linear  movement  of  the  motion  platform  and  G- 
Seat  for  training  effectiveness, 

Motion-Vision  Interaction 


This  area  of  research  is  concerned  with  determining  the  combined 
effects  upon  training  effectiveness  of  the  vision  and  motion  character isti 
of  the  device. 

Feedback 


This  area  is  concerned  wi  + h the  provision  of  feedback  (knowledge  of 
results)  to  the  student  about  .is  performance.  Usually  feedback  Is  given 
relevant  to  some  criterion  or  standard.  This  area  can  be  divided  into 
categories  as  follows: 

o Freeze  Capability  - Refers  to  the  most  effective  use  of  the 
simulator  "freeze"  capability  In  providing  feedback. 

o Sensory  Channel  - Refers  to  the  determination  of  the  most 
appropriate  sensory  channel  for  use  in  providing  feedback 
or  knowledge  of  results  to  the  student-, 

o Instructor  Provided  - Refers  to  the  deferminat ion  of  what 
I ‘ormat  i on  should  be  given  to  the  student  by  the  instructor 
as  to  the  quality  of  his  performance  based  upon  Information 
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he  obtains  either  from  his  observations  of  the  student's 
activities,  panel  Instruments,  contact  scene  or.  in  the  case 
of  the  simulator,  specific  performance  measures  of  both 
criterion  and  diagnostic  nature. 

o Frequency  - Refers  to  tho  frequency  with  which  feedback  should 
be  given  to  the  student,  i.e.,  continuous  or  at  discrete 
intervals  during  his  practice  of  the  task. 

o Time  Delay  - Refers  to  the  time  interval  between  the  performance 
of  the  task  and  the  presentation  of  information  relative  to 
the  adequacy  or  quality  of  the  performance,  i.e.,  whether  feed- 
back is  immediate  or  delayed. 

o Visual  Modeling  - Refers  to  the  process  of  exhibiting  to  the 
student  on  a visual  display  his  progress  through  a maneuver 
as  a developing  two-dimensional  spatial  pattern  on  a visual 
display.  For  example,  while  the  student  is  performing  a 
holding  pattern  using  information  derived  from  the  Instruments 
on  the  aircraft  panel  a CRT  display  may  graph  for  him  his 
ground  path  relative  to  the  desired  holding  pattern.  This 
Information  may  be  withdrawn  as  trainee  learning  progresses. 

o Self-Confrontation  - Refers  to  the  playback  of  a trainee's 
performance  during  a practice  trial  through  activation  of  the 
instruments  and  controls  of  the  simulator.  Playback  may  be 
either  in  slow,  real  or  fast  time. 

Disorientation  Training 

This  category  is  concerned  with  the  research  into  training  to  recog- 
nize and  cope  with  disorientation  during  flight.  It  Investigates  the 
requirements  for  a simulator  to  induce  disorientation  and  the  methods  and 
techniques  for  training  in  recognizing  and  coping  with  the  problem. 

Auditory  Cue  Simulation 

This  area  of  research  is  concerned  with  the  auditory  spectra  emanating 
from  the  aircraft  which  may  be  important  in  training  in  several  different 
ways  as  follow.,: 

o As  an  information  source  which  the  student  uses  in  directing 
his  control  movements, 

o As  a means  of  adding  realism  to  the  training  situation. 

o As  noise  which  interferes  with  obtaining  information  or 
which  may  otherwise  be  detrimental  to  performance. 
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Kinesthetic  Cue  (Control  Feel)  Simulation 


This  are*  of  research  deals  with  the  Investigation  of  the  degree  to 
which  the  con.r,|  forces  and  displacements  In  the  aircraft  being  simulated 
should  be  represented  In  the  simulator. 

Contextual  Training 

This  area  of  concern  Is  with  training  taking  place  within  the  context 
of  operational  tasks  and  applications.  For  example,  baste  Instrument 
maneuvers  such  as  30°  bank  turns  could  be  taught  in.the  context  of  an 
overall  maneuver  such  as  an  Instrument  approach.  An  experimental  question 
In  this  area  might  Then  be,  should  tasks  such  as  30°  bank  turns  be 
practiced  separately  and  uniquely  or  within  the  context  of  a broader 
maneuver? 

Performance  Measurement 


Within  this  category  there  are  five  identifiable.  Independent  sub- 
categories as  follows. 

o Check-ride  Performance  - Extent  to  which  check  or  criterion 
rides  In  the  simulator  may  be  substituted  for  those  in  the 
a l rcrafr. 

o Diagnostic  Measures  - Measures  of  specific  Items  at  a 

oarticular  time  during  a practice  session  or  learning  process 
which  are  designed  to  Identify  and  Isolate  specific  sources 
of  difficulty  being  experienced  by  the  trainee  In  his 
acquisition  of  the  criterion  skills  or  knowledge. 

o Control  Output  Measures  - Identification  of  the  parameters 
to  be  sampled  and  recorded,  e.g.,  lateral  stick  Input,  and 
the  methods  for  summarizing  or  transforming  them  Into  the 
most  meaningful  and  valid  form,  e.g.,  spectral  density 
function  or  frequency  analyses, 

o System  Output  Measures  - Identification  of  those  parameters 
to  be  sampled  and  recorded  by  maneuver  or  task,  e.g.,  altitude 
or  airspeed,  and  the  methods  for  summarizing  them  Into  the 
most  meaningful  and  valid  form,  e.g.,  integrated  absolute 
error,  RMS,  etc. 

o Observer  Records  of  Performance  - Investigation  of  the  methods 
and  techniques  whereby  the  instructor  pi  lot  or  other  observer 
may  record  performance  through  watching  the  standard  Instruments 
of  the  aircraft  panel,  the  contact  world  scene  and  the  actions 
of  the  trainee. 

o Observer  Opinion  Data  - investigation  of  the  methods  and 
techniques  whereby  an  observer  such  as  an  instructor  pilot. 
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test  pi  Jot,  design  engineer  or  behavioral  scientist  may 
record  or  express  his  judgments  or  opinions  in  a useful 
and  reliable  manner, 

o Determination  of  the  relationship  between  specific  and 
. detailed  diagnostic  and  criterion  measures  obtainable  in 
the  simulator  and  those  measures  possible  of  being  obtained 
by  the  instructor  pilot  observing  the  trainee  performance 
in  the  aircraft, 

instructional  Aids 

This  area  of  research  is  concerned  with  the  methods,  procedures  and 
part-task  devices  which  may  be  used  in  connection  with  a simulator  and 
within  a training  system  to  bring  about  more  effective  training.  The  list 
of  items  which  may  be  broken  out  under  this  category  are  as  follows: 

o Maneuver  Demonstration  - Refers  to  the  capability  of  the  device 
for  "playing  through"  the  maneuver  in  automated  mode  for  thi 
purpose  of  demonstrating  It  to  the  student, 

o Instructional  Cues  - In  this  item  of  research  use  is  made  of 
the  instructional  cues  derived  from  task  analyses  in  that 
the  instructional  cuas  for  each  task  or  maneuver  are  singled 
out  and  made  explicit  for  the  trainee.  Instructional  cues 
are  defined  as  the  stimuli  which  provide  the  information, 
often  in  the  form  of  a rule  or  a set  of  procedures,  which 
enables  the  learner  to  perform  the  behavior  described  in  a 
performance  objective.  It  is  the  minim?'  informational  stimulus, 
either  audible,  visual  or  tactual,  which  must  be  supplied  to 
the  learner  in  order  to  enable  him  to  make  the  desired  response. 
The  emphasis  In  this  area  of  research  would  be  the  pointing 
out  or  making  explicit  to  the  learner  what  Instructional  cues 
are  involved  In  performing  the  task.  This  may  be  carried  out 
through  use  of  the  visual  and/or  auditory  display  capabilities 
of  the  ASUPT  or  use  of  other  less  complex  audio  and/or  video 
devices, 

o Part-Task  Trainers  - Research  using  much  less  complex  devices 
than  the  ASUPT  to  train  in  parts  or  aspects  of  a task  prior 
to  ASUPT  training.  Such  devices  might  include  full  scale 
panel  illustrations  or  pictures,  mock-ups,  procedures,  trainers 
and  audio-visual  devices  for  training  in  procedural,  Derceptual 
and  cognitive  type  tasks. 

o Prompting  and  Cueing  - Prompts  are  signals  whfch  indicate  that 
the  time  has  come  for  a specific  action  to  occur  and  directs 
the  -student  to  perform  that  particular  action.  Cueing  is 
similar  to  prompting  but  usually  refers  to  a simple  signal 
that  indicates  time  to  act.  Thus,  a cue  is  less  directive 
than  a prompt. 
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Aircraft  Dynamics  Simulation 

This  area  of  research  deals  with  the  degree  to  which  the  equations 
describing  the  motions  of  the  simulated  aircraft  should  be  represented 
and  implemented  in  the  simulator.  It  Is  directed  essentially  toward 
the  degree  to  which  these  eauations  might  be  simplified  in  their 
implementation  in  the  simulator  with  possible  reduction  in  required 
computer  speed  and/or  capacity. 

Peer  Training 

Research  in  which  the  trainee's  peers  may  observe  and  work  with  him 
while  he  practices  or  engages  in  the  solution  of  problems.  This  category 
may  be  subdivided  as  follows: 

o Instructional  Assist  - In  which  the  fellow  trainee  engages 
in  helping  the  trainee  solve,  problems  or  improve  his  skills 
through  discussion,  suggestion  and  critiaue  of  his 
performance. 

o Dynamic  Observer  - in  which  the  fellow  trainee  acts  as  an 
observer  and  notes  the  performance  of  the  trainee 
being  observed,  the  types  of  errors  which  develop,  and 
can  mentally  rehearse  his  own  techniaues  and  approach  to 
solving  the  problem  or  acquiring  the  skill. 

Instructor's  Role  and  Training 

This  area  of  research  is  concerned  with  developing  information  which 
may  be  used  to  make  the  instructor  more  effective  in'his  guidance  and 
management  of  the  student's  progress  through  the  training  curriculum. 

It  may  be  subdivided  into  the  following  categories: 

o Performance  Evaluation  - This  area  includes  research  in  the 

training  of  the  instructor  in  the  evaluation  of  the  performance 
of  the  trainee  in  the  simulator  and  in  the  aircraft  and  the 
relationship  between  the  performance  measures  obtained  in  each 
of  these  training  situations,  A greater  array  of  performance 
evaluation  means  is  available  to  the  instructor  in  the 
simulator  than  in  the  aircraft.  Instructor  training  in  the 
use  of  diagnostic  and  criterion  proficiency  measurements 
in  the  simulator  is  the  concern  of  this  research  area.  Of 
particular  concern  is  the  Interpretation  and  translation  of 
the  more  detailed  measures  taken  in  The  simulator  into 
instructional  guidance  and  performance  evaluation  in  the 
ai rcraft. 


o Use  of  the  diagnostic  information  provided  by  the  simulator 
performance  measures  for  eitner  extemporaneous  or 
standardized  instructions  and  guidance  to  the  trainee. 
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o Oegres  of  Instructor  Participation  - This  research  Item 
refers  to  the  degree  to  which  the  Instructor  actively 
participates  In  the  ongoing  practice  of  the  trainee.  On 
the  one  hand  the  Instructor  may  demonstrate  the  task, 
talk  the  student  through  maneuvers  and  provide  evaluative 
and  directive  Information  throughout  the  course  of  the 
student?s  practice.  On  the  other,  he  may  adopt  a passive 
role  In  which  he  provides  Information  only  when  questioned 
by  the  trainee  and  Injects  himself  Into  the  training 
practice  only  to  assure  safety. 

o Instructor  Motivation  - Investigation  of  the  Incentives  and 
awards  which  may  be  used  to  Increase  Instructor  Interest  and 
enthusiasm! 

o Instructor  Standardization  - Research  concerned  with  training 
and  measuring  the  instructor's  ability  such  that  he  maintains 
his  effectiveness  at  a defined  and  established  level. 

o Degree  of  Instructor  Task  Automation  - This  Item  refers  to 
the  analysis  of  the  Instructor's  task  and  the  allocation  of 
certain  of  these  to  automatic  execution  by  the  training 
simulator.  The  types  of  functions  and  tasks  that  fall 
within  this  category  are  briefings,  demonstrations, 
performance  evaluation  and  assignment  of  training  tasks. 

o Instructor  as  Training  Manager  - Refers  to  the  Instructor's 
control  of  the  progression  of  the  student  through  the  syllabus 
based  upon  student  performance  relative  to  specified  criteria 
• using  specific  task  descriptions  and  criteria  and  having 

available  to  him  detailed  knowledge  of  the  student's  progress 
through  diagnostic  as  well  as  criterion  performance  measurement, 

\ o Instructor  Station  and  Role  in  UPT  Simulation  - Refers  to  such 
\ questions  as  whether  the  instructor  should  be  located  In  or  out 
of  the  cockpit  and  the  optimum  design  of  the  instructor's 
console. 

Relationship  of  Trainee  Traits  and  Methods  of  Instruction 


This  area  of  research  is  concerned  with  the  investigation  of 
relationships  between  methods  of  presenting  the  training  materials  and 
certain  measurable  traits  of  the  individual.  For  example,  measurable 
traits  such  as  visual  field  dependency,  manifest  anxiety  or  perceptual 
and  cognitive  style  may  be  hypothesized  to  be  related  to  the  appropriate 
method  or  mode  of  instruction  to  be  used  for  a given  trainee. 

Cognitive  Pre-Training 

This  area  refers  to  extensive  briefing  or  comrjlete  understanding 
of  the  perceptions  and  actions  required  for  completing  a task  before 
it  is  attempted  in  the  simulator  or  the  aircraft. 


Sequencing  of  Training  Tasks,  Maneuvers  and  Phases 


This  area  of  research  is  concerned  with  the  ordering  of  the  tasks, 
maneuvers  and  phases  such  that  optimum  transfer  from  fundamental  skills 
and  knowledges  to  tasks  of  greater  difficulty  is  obtained  and  results 
In  the  most  efficient  progression  through  the  syllabus.  Specific  research 
questions  may  range  from  whether  Instrument  flight  training  should 
precede  or  follow  contact  flight  training  to  investigations  which  seek 
to  determine  the  hierarchy  of  skills  In  the  optimum  order  and  methods 
of  teaching  these  basic  skills. 

Trainee  Motivation 


. This  area  of  research  involves  Investigation  of  the  role  which  in- 
centives may  play  in  bringing  about  more  effective  training.  These  may 
involve  competion  among  trainees  or  other  Incentives  and  awards  for 
accompi I shment. 

Extension  of  the  Training  Syllabus 

This  area  is  concerned  with  determining  the  emphasis  to  be  placed 
upon  training  tasks  and  maneuvers  which  are  avoided  in  the  aircraft  for 
reasons  of  safety,  and  those  which  require  controlling  the  aircraft  to 
the  limits  of  Its  performance  and  structural  integrity,  e.g.,  control 
at  the  critical  airspeed  limits. 

Adaptive  Training 

In  the  simulated  environment,  UPT  may  be  adapted  in  various  ways. 
Under  either  automated  or  manual  control,  the  actual  task,  the  syllabus, 
and/or  simulator  characteristics  C3n  be  modified.  The  specific  Items 
in  this  area  are: 

o Individualized  Instruction  - Determination  and  description  of 
the  specific  tasks  to  be  taken  up  in  the  syllabus,  specification 
of  criterion  performance  either  normative  or  administrative, 
and  progression  of  the  student  through  the  syllabus  on  a 
performance  to  criterion  basis  with  provisions  for  branching 
back  for  rehearsal  of  tasks, 

o Machine  Adaptive  Training  - Adjustment  of  the  characteristics 
of  the  training  device  or  media  or  task  such  that  the  trainee 
progresses  to  criterion  performance  at  his  own  individual  pace 
based  upon  measurement  of  his  progress.  Adjustment  of  the 
training  situation  is  customarily  such  that  the  trainee's  task 
is  easy  during  his  Initial  practice  trials  and  is  adjusted  to 
. become  difficult  based  upon  his  progress  until  he  reaches 
criterion  performance. 
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APPEND »X  C 


Life  Sciences,  Inc. 
Contract  F4I609-73-C-0038 


Assignment  of  Priorities  to 
Training  Problems 

The  attached  list  dtscribes  areas  of  research  which  have  been  identi- 
fied as  those  which  are  pressing  in  terms  of  their  Importance  for  making 
pilot  training  more  effective.  Research  results  providing  definitive  infor- 
mation in  these  areas  would  be  highly  useful  In  making  decisions  as  to  the 
way  training  should  be  conducted  and  the  type  of  training  equipment  that 
should  be  used. 

What  Is  needed  now  Is  an  ordering  of  the  listed  Items  In  terms  of 
their  Importance  to  Increasing  the  effectiveness  of- beginning  or  under- 
graduate pilot  training.  That  parrlcular  Item  which  the  rater  feels  should 
be  taken  up  first  in  an  undergraduate  pilot  training  research  program  should 
appear  at  the  top  of  the  list.  The  basis  for  assigning  priorities  should 
be  the  Importance  of  the  research  Item  for  Its  potential  for  Increasing 
training  effectiveness.  Increasing  training  effectiveness  Is  defined  as 
bringing  the  student  to  criterion  proficiency  in  less  training  time  at  less 
training  cost. 

The  question  posed  to  the  rater  is  - what  is  the  particular  order  In 
which  the  Items  of  research  should  be  accomplished  in  order  to  provide 
the  most  useful  information  to  administrators  in  making  training  program 
decisions.  It  is  hoped  that  a reasonably  concise  and  unamblglous  definition 
of  each  of  the  research  areas  has  beer*  given  so  that  they  may  be  reviewed 
and  priorities  assigned. 

Each  of  the  items  has  been  listed  separately  on  a 3 x **  card  so  that 
they  may  be  sorted  and  arranged  in  order  of  priority.  As  you  receive 

them  the  cards  will  b8  in  the  same  order  as  the  items  in  the  attached  list. 

The  procedure  for  sorting  Is  to  select  a given  card  and  place  It  face  up 
before  you.  Each  of  the  other  Items  (cards)  is  examined  and  a judgment 
made  as  to  whether  it  is  of  greater  or  lesser  priority  than  the  first 
chosen  (reference)  card.  Any  item  which  is  judged  to  be  of  higher  priority 

than  the  reference  card  is  placed  to  the  right  of  ti.at  card  while  those 

judged  to  be  of  lesser  priority  are  placed  to  the  left.  At  the  conclusion 
of  this  sort  the  right  hand  stack  of  cards  is  taken  up  and  sorted  in  pre- 
cisely the  same  manner.  That  is,  a reference  card  Is  chosen  and  placed 
face  up  before  the  rater.  The  remaining  cards  in  the  stack  are  then  sorted 
into  two  piles,  one  on  either  side  of  the  reference  card  designating  either 
higher  priority  or  lower  priority  thar-  the  reference  card.  This  procedure 
Is  repeated  f or  all  stacks  urJ  I no  cna  has  more  than  three  or  four  item 
cards  in  it.  These  three  c*  four  may  then  be  ranked  from  high  to  low  in 
priority.  The  total  stack  of  cards  is  then  reassembled  from  greatest  to 


least  priority.  It  ft  Is  convenient  the  Items  should  be  typed  Into  a list 
In  the  priority  order  shown  by  the  cards.  If  this  Is  hot  convenient,  the 
prioritized  card  stack  should  be  secured  firmly  and  returned  to  Life 
Sciences  in  this  form.  It  is  preferred,  both  for  convenience  In  mailing 
and  for  insuring  that  the  Items  will  not  be  gotten  out  of  order,  that  the 
typed  list  be  made. 
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APPENDIX  D 


Ranking  of  Training  Problems  Based 
Upon  Median  Rank  (10  Raters)' 


ITEM 

MEDIAN 

Q 

1. 

Content  of  the  Visual  Display 

6.0 

10.75 

2. 

Motion-Vision  Interaction 

1 1.0 

11.50 

3. 

Quality  of  the  Visual  01  splay 

12.5 

19.75 

4. 

Performance  Measurement -System  Output 
Measures 

12.5 

12.75 

5. 

Sequencing  of  Training  Tasks 

14.0 

11.00 

6. 

Contribution  of  the  Individual  or 
combined  Degrees  of  Freedom  of  Plat- 
form Motion  to  Training 

14.0 

11.75 

7. 

Instructor  Training  - Performance 
Eva  1 uat 1 on 

14.5 

11.00 

8. 

Cognitive  Pre-Training 

15.0 

9.75 

9. 

Performance  Measurement  - Diagnostic 
Measures 

18.0 

10.00 

10. 

Adaptive  Training  - Individualized 

Instruction 

18.0 

7.50 

H. 

Feedback  - Sel f-Confrontatlon 

20.0 

14.25 

12. 

Feedback  - Visual  Modeling 

* 20.0 

8.00 

13. 

Instructor  Training  - Use  of  1 agnos- 
tic Informat' on  for  Student  Guidance 

21.0 

12.00 

14. 

Feedback  - Sensory  Channel 

21.5 

18.75 

15. 

Instructional  Aids  - Instructional 

Cues 

21.5 

8.50 

«6, 

Cori  * uc.« . i j*a  i n l ng 

22.0 

9.00 

17. 

Contribution  of  the  Individual  or 
Combined  Degrees  of  Freedom  of  Platform 
Motion  to  Proficiency  Measurement 

24.0 

17.50 

18. 

Instructor  Training  - Degree  of 
Instructor  Participation 

24.5 

18.00 

19. 

Feedback  - Instructor  Provided 

24.5 

18.00 

20. 

Field  of  View 

25.0 

16.75 

21. 

Adaptive  Training  - Machine 

25.5 

10.75 

22. 

Determination  of  the  Optimum  Fidelity  of 
Response  in  Pitch,  Roll  and  Yaw  of 
the  Motion  Platform 

25.5 

16.00 

23. 

Instructor  Motivation 

26.0 

23,75 

24. 

Performance  Measurement  - Relationship 

27.5 

9.50 

25. 

Determination  of  the  Degree  to  Which 
G-Seat  May  Provide  Motion  Cues 

27.5 

7.75 

26. 

Feedback  - Frequency 

28.5 

12.75 

27. 

Performance  Measurement  - Observer 
Records  of  Performance 

28.5 

13.25 
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Ranking  of  Training  Problems  Based 
Upon  Median  Rank  (Cont’d) 


MEDIAN 


28. 

Determination  of  the  Optimum  Program  for 
Linear  Movement  of  the  Motion  Platform 

29.0 

13.75 

29. 

Determination  of  the  Importance  of  the 
Auditory  Spectra  as  an  Information 
Source 

29.0 

16.00 

30. 

Instructor  Role  as  Training  Manager 

29.5 

16.25 

31 . 

Peer  Training  - Instructional  Assist 

29.5 

11.75 

1*2. 

Performance  Measurement  - Control 
Output  Measures 

30.0 

14.25 

33. 

Instructional  Aids  - Part-Task 
Trainers 

31.0 

14.0 

34. 

Depth  of  Field  of  View  in  the  ' 

Visual  Display 

31.5 

18.50 

35. 

Instructional  Aids  - Prompting  and 
Cueing 

32.0 

12.25 

36. 

Instructor  Standardization 

32,0 

16.25 

37. 

Performance  Measurement  - Checkride 
performance 

32.5 

10.00 

38. 

Determination  of  the  Contributions  to 
Training  of  the  Sustained  Pressures 
Provided  by  the  G-Seat 

33,0 

17.50 

39. 

Degree  to  Which  the  Control  Forces 
and  Displacements  Should  be  Represented 
In  the  Simulator 

33.5  . 

11,50 

40. 

Relationship  of  Trainee  Traits  and 
Methods  of  Instruction 

33.5 

20.75 

41. 

Instructor  Training  - Degree  of 
Instructor  Task  Automation 

34,0 

19.50 

42. 

Peer  Training  - Dynamic  Observer 

34.0 

13.25 

43. 

Instructional  Aids  - Maneuver 
Demonstration 

35.0 

15.50 

44. 

Extension  of  the  Training  Syllabus 

35.0 

14.75 

45. 

Contribution  of  the  Gravity  Alignment 
Cue  to  Training 

35.0 

16.25 

46. 

Disorientation  Training 

35.5 

11.25 

47. 

Feedback  - Time  Delay 

36.0 

10.00 

48. 

Trainee  Motivation 

36.5 

20.50 

49. 

Performance  Measurement  - Observer 
Opinion  Data 

38.0 

8.00 

50. 

Determination  of  the  Importance  of  the 
Auditory  Spectra  as  a Means  of  Adding 
Realism  to  the  Training  Situation 

40.0 

' 5.50 

51. 

Determination  of  the  Importance  of  the 
Auditory  Spectra  as  Interference  or  Noise 

44.0 

10.75 

52. 

Aircraft  Dynamics  Simulation 

45.0 

18.75 

Ranking  of  Training  Problems  Based 
Upon  Median  Rank  (Cont’d) 


ITEM 

MEDIAN 

0 

53. 

Feedback-Simulator  Freeze  Capability 

47.0 

7.00 

54. 

Color  In  the  Visual  Display 

47.5 

10.25 

55. 

Instructor  Station  » Location  and 
Design 

48.0 

7.25 

Note  I. 

In  this  table  the  median  refers  to  the  mld-polnt  In  the  distribution 
of  ratings  given  by  the  judges*  it  is  that  rating  value  at  which  one-half 
or  50>  of  the  judges  ratings  of  the  particular  Item  were  greater  than  the 
value  and  one-half  of  them  were  lower.  The  Q value  In  the  table  Is  a 
measure  of  the  variability  of  the' judges’  ratings  and  Is  called  the  seml- 
Interquartl le  range.  The  formula  for  cue  Is  Q = Q3  - Qf.  0|  is  the  point 
one-quarter  through  the  distribution  of  ratings,  l.e.,  the  point  at  which 
75J  of  the  ratings  are  above  this  value  and  25£  are  balow.  O3  is  the  point 
three-quarters  of  the  way  through  the  distribution.  Thus,  the  difference 
between  O3  and  Q|  divided  by  two  is  a measure  of  the  spread  or  variability 
of  the  judges’  ratings. 
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APPENDIX  E 

Scenario  for  Flights  of  Instrumented  T-37  Aircraft 
Sortie  No.  1 

Record  runway  barometric  pressure  and  temperature,  date,  time. 
Take-off. 

Turn  on  recorder. 

Gear-up  - record,  (I)  event  no.,  (2)  fuel  quantity,  (3)  turbulence 
level,  (4)  "climb”. 

Climb  to  15,000  feet  at  a constant  180  KIAS.  Perform  4-2  minute 
trials  during  climb.  Trim  before  each  trial  as  necessary. 

Fly  to  area  where  maneuvers  can  be  flown. 

Update  event  mark  - record,  (I)  event  no.,  (2)  fuel  quantity,  (3) 
turbulence  level,  (4)  "descent". 

Descend  at  a constant  airspeed  of  200  KIAS  to  2,000  feet.  Perform 
4-2  minute  trials  during  descent.  Trim  before  each  trial  as 
necessary. 

Trim  to  200  KIAS,  straight  and  level,  2,000  feet. 

Record  (i)  event  no.,  (2)  fuel  quantity,  (3)  turbulence  level, 

(4)  trial  order  A. 


Fly  2 minute  trials  In  order  A.  Trim  before  each  trial  as 
necessary. 

Record  (I)  event  no.,  (2)  fuel  quantity,  (3)  turbulence  level, 
(4)  "straight  and  level  to  base". 

Return  to  base  at  2,000  feet.  Perform  2 minute  trials  straight 
and  level  during  return.  Trim  before  each  trial  as  necessary. 


SorMe  No.  2 

1.  Record  runway  barometric  pressure  and  temperature,  date,  time. 

2.  Take-off. 

3.  Turn  on  recorder. 

4.  Gear-up  - record,  (I)  event  no.,  (2)  fuel  quant!'. y,  (3) 
turbulence  level,  (4)  "climb". 

3.  Climb  to  15,000  feet  at  a constant  180  KIAS.  Perform  4-2 

minute  trials  during  climb.  Trim  before  each  trial  as  necessary, 

6.  Fly  to  area  where  maneuvers  can  oe  flown. 

7.  Record,  (I)  event  no.,  (2)  fuel  quantity,  (3)  tur5ulence  level, 

(4)  "descent”. 

8.  Descend  at  a e.  stant  airspeed  of  200  KIAS  to  2,000  feet. 

Perform  4-2  minute  trials  during  descent.  Trim  before  each 
trial  as  necessary. 

9.  Trim  to  200  KIAS,  straight  and  level,  2,000  feet  above  ground. 

10.  Recoro  (I)  event  no.,  (2)  Tuel  quantity,  (3)  turbulence  level, 

(4)  trial  order  B. 

11.  Fly  2 minute  trials  in  order  B.  Trim  before  each  triat  as 
necessary. 

12.  Record  (I)  event  no, . (2)  fuel  quantity,  (3)  turbulence  levei, 

(4)  "straight  and  level  to  base". 

13.  Return  to  base  =*t  2,000  feet.  Perform  3-2  minute  trials  straight 
and  level  during  return.  Trim  before  each  trial  as  necessary. 
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Sorties  344 


Repeat  Sorties  No.  I and  2 except  perform  straight  and  level  and  30° 
bank  turns  at  100  KIAS. 


Sorties  546 


Repeat  Sorties  No.  I and  2 except  perform  straight  and  level  and  30® 
bank  turns  under  contact  conditions. 


Sorties  743 


Repeat  Sorties  No.  I and  2 except  perform  straight  and  level  and  30° 
bank  turns  are  at  100  KIAS  and  under  contact  conditions. 
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APPENDIX  F 

Eye-movement  Record! nqs  and  Data  Analysis 

It  Is  recommended  that  the  NAC  Eye  Mark  Recorder  Is  to  be  used  as  a 
method  for  gathering  data  for  the  Identification  of  the  objects  comprising 
the  content  of  the  visual  display  which  can  be  used  as  referents  by  the 
pilot  In  control  of  his  aircraft.  The  NAC  Eye  Mark  Recorder  provides  a 
record  of  a primary  field  of  view  60°  In  azimuth  of  the  scene  directly 
forward  of  the  pilot’s  head  position  and  an  Indication  on  this  flelo  » 
view  of  the  pllof’s  fixation  point  within  the  scene.  Specifically,  fh«.» 
recorder  Is  an  optical  device  which  focuses  an  Illuminated  reticle  re- 
flected by  the  movement  of  the  eyeball  so  that  It  always  coincides  with  the 
visual  1 1 ne-of- regard.  This  is  superimposed  on  the  primary  jmage  and  may 
be  recorded  on  16mm  film  or  video  tape.  The  video  tape  recordlnq  method 
is  recommended  because  of  Its  capability  for  Instant  replay. 

The  general  approach  recommended  for  the  collection  of  these  data  Is 
to  require  Instructor  pilots  to  fly  representative  undergraduate  pilot 
training  maneuvers  while  their  eye  movements  are  recorded.  The  specific 
maneuvers  recommended  to  be  flown  are  (a)  straight  and  level,  (b)  steep 
turn,  (c)  complete  (deep)  stall,  (d)  lazy  eights,  (e)  a shallow  turn,  (f) 
pattern,  approach  and  landing,  and  (g)  formation  flight. 

Instructor  pilot  subjects  will  be  selected  who  are  current  In  the 
maneuvers  to  be  accomplished  during  data  collection.  These  pilots  will  be 
briefed  on  the  general  characteristics  of  the  eye  mark  recording  equipment 
and  such  safety  procedures  as  are  required.  This  briefing  will  Include 
Information  as  to  the  purpose  of  the  maneuver  and  data  col  lection  procedure. 
Thj  pilots  will  be  informed  as  to  why  they  are  being  monitored  and  that  at 
conclusion  of  a flight  they  will  be  asked  to  review  the  recordings  of 
thiir  eye  movements  and  to  relate  what  elements  of  the  visual  scene  they 
were  using  as  they  piloted  their  aircraft. 

An  important  phase  of  this  investigation  is  the  post  flight  review 
by  the  pilot  of  his  eye-movement  recordings.  It  is  necessary,  on  a moment- 
by-moment  basis,  for  the  subject  pilot  to  identify  the  particular  element 
In  the  contact  world  scene  which  he  Is  using  to  provide  him  with  Infor- 
mation for  control  of  specific  dimensions  of  movement  of  his  aircraft,  it 
is  also  necessary  that  he  identify  any  relationships  either  static  or 
dynamic  between  the  external  world  referents  and  those  fixed  to  his  air- 
craft. These  referent  points  will  serve  to  Identify  objects,  points,  lines, 
features  or  characteristics  which  may  then  be  generated  in  the  contact 
real --world  scene  provided  by  the  simulator.  These  Idontlfled  referents 
will  provide  the  basis  for  Investigations  as  to  the  content  objects  and 
their  location  in  subsequent  research  investigations. 

It  is  also  hypothesized  that  It  Is  a requirement  of  the  pilot  trainee 
to  learn  to  Identify  and  extract  the  pertinent  referen.s  from  visual 
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"noise"  In  the  contact  world  scene.  This  noise  is  comprised  of  other 
objects  or  conditions  which  serve  to  obscure  the  referent  and  Interfere 
with  fine  discriminations  of  its  relaflonshlp  to  other  referents.  During 
the  post  flight  interview  the  pilot  must  be  asked  whether  he  can  identify 
such  condition  in  the  videotape  he  reviews.  They  may  be  such  factors  as 
haze,  atmospheric  attenuation,  iow  contrast,  or  any  other  which  tended 
to  interfere  with  or  degrade  his  visual  perceptions. 


Of  particular  interest  in  the  post-flight  interview  Is  information 
relevant  to  the  peripheral  cues  observed  and  used  by  the  pilots  during  the 
flight.  The  long  standing  belief  in  the  importance  of  peripheral  cues 
(Hasbrook  & Young,  1968)  has  been  recently  reinforced  by  data  collected 
by  Life  Sciences,  Inc,  in  which  pilots  eye-movement  records  were  taken 
while  hovering  the  helicopter.  These  records  show  that  a pilot  could 
fixate  steadily  on  a given  point  on  the  ground  plane  for  a long  period  of 
time  while  controlling  all  six  dimensions  of  movement  of  his  aircraft  In- 
dicating great  reliance  on  peripheral  visual  information. 


Although  the  NAC  Eye  Mark  Recorder  is  equipped  with  a 60°  field  of 
view  lens  the  peripheral  vision  of  the  human  operator  extends  beyond  that 
field  and  it  is  quite  probable  that  information  is  available  and  utilized 
by  the  pilot  which  is  not  recorded.  The  post-flight  review  of  the  record- 
ings and  the  interviews  should  cover  this  possibility,  i.e.,  the  pilot 
should  attempt  to  recall  and  elaborate  upon  any  peripheral  cues  which  are 
not  visible  in  recordings  at  any  point  he  feels  necessary. 


Although  the  I ine-of-regard  of  the  pilot  can  be  established  with  the 
Eye  Mark  Recorder  this  does  not  establish  the  fact  of  the  object  or  area 
on  which  the  eye  is  focused.  That  is  to  say,  that  the  I ine-of-regard 
reticle  of  the  recorder  resting  upon  an  object  dees  not  necessarily  mean 
that  the  eye  is  focused  upon  that  object,  although  it  is  highly  probable 
that  it  is.  T!a  pilot,  therefore,  must  be  able  to  explain  precisely  what 
object  or  point  he  was  focused  upon  during  the  portion  of  the  maneuver 
which  is  visible  on  the  video  tape.  The  pilot  may  provide  narrative  infor- 
mation on  a voice  tract  of  the  video  tape  during  data  collection.  This 
narrative  is  not  deemed  critcally  necessary  but  could  prove  helpful. 

If  there  is  any  Indication  that  such  a narration  Is  interferring  with 
the  pilots  norma!  flying  of  the  maneuver  it  should  be  discontinued. 


Although  it  is  preferable  that  a given  maneuver  be  accomplished  in 
"discrete"  fashion  with  Immediate  return  to  the  flight  line  for  playback 
and  Interview,  the  data  collection  sequence  is  open  to  variations  because 
of  the  practicalities  of  flight  data  collection.  Tor  example,  data  might 
be  collected  on  taxi  and  take-off,  ciimbout,  and  Immediate  approach  and 
landing  with  return  to  the  flight  line  and  playback  and  interview  covering 
this  sequence,  it  is  recommended,  however,  that  to  the  maximum  extent 
possible,  an  extended  sequence  of  maneuvers  not  b8  accomplished  since  It 
would  require  a greater  amount  of  recall  by  the  pilot  and  certain  factors 
may  be  forqo+ten  before  the  interview  session. 


Documentation  of  eye  point  data  in  the  total  field  of  view  will  be  of 
direct  assistance  in  establishing  the  visual  environment  of  the  pilot  and 
can  be  utilized  in  other  programs.  For  example,  the  data  could  be  used 
by  instructor  pilots  for  directing  the  student's  attention  tc  particular 
areas  of  the  visuai  field  during  training.  However,  first  and  primary  use 
of  the  records  and  the  interview  data  is  in  generating  hypotheses  as  to 
the  visual  Image  content  and  fields  of  view  to  be  tested  In  ASUPT. 


The  outline  of  the  procedure  for  carrying  out  the  eye-movement  data 
collection  and  analysis  is  as  foi lows: 


*o  Purchase  NAC  Eye  Mark  Recorder  and  modify  for  helmet  mounting 


*o  Preliminary  try-out  of  equipment  and  procedures  with  HRL/FT 
personnel 


o Selection  of  instructor  pilot  subjects 


o General  orientation  and  safety  briefing 


o Selection  and  data  collection  on  maneuver 


o Playback  and  interview  concerning  maneuver 


o Analysis  to  determine  field  of  view 


o Formulation  of  hypotheses  for  test 


*Accomp I i shed  as  of  this  date 


At  the  time  of  the  writing  of  this  report  the  eye-movement  recording 
program  is  going  ahead  at  HRL/FT.  As  indicated  in  the  list  of  procedures 
to  be  accomplished  the  NAC  Eye  Mark  Recorder  was  obtained  and  modified 
for  helmet  mount.  Preliminary  data  collection  to  try  out  both  the  Eye 
Mark  Recorder  and  the  video  tape  equipment  has  been  accomplished  using 
ground  vehicles.  The  means  for  mounting  and  operating  the  eye  mark 
equipment  and  the  recorder  in  the  TF37  aircraft  have  been  specified. 
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APPENDIX  G 


Functional  Description  of.AVIT  Device 


A schematic  drawing  of  the  existing  AVIT  Is  given  in  Figure  1. 


The  stimulus  devices  to  which  student  responds  are: 


I.  An  18  x II  Inch  rear-projection  screen  on  which  photos  of  the 
basic  flight  Instrument  panel  of  "the  T-37,  or  other  photographic,  written, 
or  diagrammatic  material,  as;  appropriate,  may  be  presented; 


2,  A head-set  for  audio  messages; 


3.  A set  of  six  "response  evaluation"  lights  In  two  rows  of  three, 
the  top  row  labelled  RO.LL  OK,  PITCH  OK,  POWER  OK  against  a green  back- 
ground, and  the  bottom  row  labelled  ROLL  ERROR,  PITCH  ERROR,  POWER  ERROR 
against  a red  background. 


The  response  mechanisms  that  the  student  may  use  ana: 


stick. 


I.  A simulated  stick  :'.d  throttle,  with  a "trigger"  switch  on  the 


2.  A row  of  5 response  buttons,  label  !<sd  A , C,  D,  and  R. 


In  addition  to  the  above  the  student  hiss  available  a SOUND  REpEAT 
button,  which  allows  him  to  repeat  m audio  rsssage,  and  an  EXPOSURE 
CONTROL  dial,  the  use  of  which  will  be  described  later. 


Trainer  Mode 


The  AVIT  operates  in  a nuT.ih  r of  different  modes,  which  are  selected 
automatically  by  the  program  material,  ' Ho  basic  TRAINER  mode  will  be 
described  first. 


Early  in  an  AVIT  progrua  a typical  program  segment  (Figure  2)  might 
be  as  follows:  on  frame  20  of  the  program,  following  some  introductory 

material,  the  student  might  see  on  the  screen  a photo  of  the  instrument 
panel  with  all  the  Instruments  showing  the  readings  that  are  correct  for 
a particular  flight  condition,  as  shown  In  Figure  3.  The  accompanying 
audio  message  (A-10)  reviews  the  readings  and  Instructs  the  student  to 
press  his  trigger  switch  to  advance  the  first  problem  frame,  frame  21. 


On  frame  21,  all  instrument  readings  might  be  normal  except  for  a 
sma'.«  Increase  In  altitude.  There  will  bo  no  sound  massage  with  frame 
21,  but  the  student  will  have  been  Instructed  previously  that  ho  is  to 
make  the  appropriate  control  movements  to  return  to  his  required  flight 
condition.  He  moves  the  controls  to  tne  position  he  desires  and  presses 
his  trigger  switch  to  "register"  his  response. 
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Figure  4«  Simple  AY  IT  Soquence,  Confirmation  Framos  Cmlttod 


If  he  has  responded  Incorrectly  the  AVIT  will  move  automatically  to 
frame  22,  The  visual  material  on  frame  22  will  be  the  same  as  on  frame  21, 
but  there  will  be  an  audio  message  <A-M)  which  will  call  attention  to  the 
specific  Instrument  readings  requiring  ccrrectlve  action.  In  addition,  the 
appropriate  response  evaluation  lights  will  come  on.  The  audio  message 
might  be: 


"Here  your  attitude,  airspeed  and  heading  are  all  right,  but  you 
are  100  feet  above  your  assigned  altitude.  Check  the  lights  to 
see  whore  your  specific  error  was  and  then  press  the  trigger  to 
return  to  try  the  problem  again". 


When  the  student  presses  the  trlggor  the  AVIT  returns  to  frame  21, 
and  the  response  evaluation  lights  go  out,  of  course. 


When  the  student  has  made  the  right  response  the  AVIT  will  move  to 
frame  23,  The  visual  presentation  on  frame  23  will  be  the  normal  set  of 
readings  as  on  frame  20,  and  the  sound  message  ( A— I 2 ) will  specifically 
confirm  the  correctness  of  the  response.  For  example.  A- 1 2 might  be: 


"Very  good.  The  only  problem  was  a 100  foot  error  on  the  altimeter. 
Forward  stick  was  needed  to  begin  a return  to  I5it000  feet.  For  this  small 
decrease  in  altitude,  no  power  change  would  be  necessary.  Very  well,  press 
the  trigger  swlfch  w;  the  next  problem." 


When  the  stuaenf  pressed  the  trigger,  the  AVIT  would  move  forward  to 
frame  23,  and  so  on. 


Uster  In  the  program,  the  confirmatory  frame  would  be  omitted.  <?nd  the 
student  making  the  correct  response  would  move  directly  to  the  next  problem 
frame.  The  green  lights  would  come  on  briefly  to  confirm  the  correctness  of 
his  response,  and  he  would  have  previously  been  Instructed  that  he  should 
respond  to  the  new  problem  directly.  Such  a program  segment  Is  shown  In 
Figure  4, 


The  program  segment  structures  just  discussed  are  of  the  simplest  kind 
and  the  AVIT  Is  capable  of  handling  much  more  complex  structures.  Figure  5 
shows  a 5-question  "pretest"  structure,  arranged  so  that  the  student  who 
passes  the  pretest  with  a perfect  score  will  automatically  skip  a particular 
instructional  sequence.  Frame  40  Is  the  introductory  frame  for  the  pretest, 
Frames  41,  42  and  43  are  a "warm  up"  problem  on  which  an  error  will  be 
"forgiven",  and  the  audio  message  on  frame  43  (A-27J  wouid  warn  the  student 
that  from  now  on  It  was  "for  real"  and  ask  him  to  press  his  trigger  to 
begin  the  test. 


In  Figure  5,  the  problem  frames  as  such  are  '-ames  44,  46,  48  and  50 
and  52,  Any  error  leads  the  student  to  frame  55,  where  the  Instruction  on 
the  particular  topic  begins.  The  student  who  gets  all  5 problems  right 
comes  to  frame  54,  The  possible  excursion  from  frame  54  to  55  represents 
tho  fact  that  we  might  offer  the  student  who  reaches  frame  54  the  option 
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of  going  through  the  Instruction  on  the  particular  maneuver  If  he  desires 
to  do  so,  even  though  he  has  passed  the  pretest.  (How  the  student  Indicates 
his  choice  on  frame  54  will  be  discussed  under  DIDACTIC  mode  later.) 

Both  students  and  Instructor-pilots  have  been  enthusiastic  about  the 
program  structure  shown  In  Figure  6,  used  In  connection  with  the  EXPOSURE 
CONTROL  mentioned  previously.  Basically,  the  structure  Is  a 7 problem 
test  sequence  where  any  error  returns  the  student  to  the  start  of  the 
sequence.  On  the  introductory  frame  (80),  however,  the  student’s  attention 
Is  called  to  the  EXPOSURE  CONTROL  switch,  which  has  the  following  settings: 
SELF-PACED,  II  secs,  8 secs,  5 secs,  3.5  secs,  and  2.5  secs.  The  dis- 
cussion so  far  has  assumed  that  this  control  was  set  on  SELF-paced,  and 
that,  therefore,  the  problem  remained  on  the  screen  until  the  student 
pressed  the  trigger  switch  to  register  his  response.  On  the  timed  settings 
the  trigger  switch  Is  automatically  pressed  (in  effect)  at  the  end  of  the 
time  set.  If  the  student  has  not  moved  the  controls  to  the  correct 
position  at  the  expiration  of  the  set  time,  he  will  score  an  error,  of 
course.  If  he  Is  correct,  he  moves  on  to  the  next  problem,  (He  can,  of 
course,  register  his  answer  in  the  normal  way,  by  pressing  his  trigger 
switch,  even  with  the  EXPOSURE  CONTROL  timer  on,  If  he  does  so  before  the 
time  expires. ) 

The  EXPOSURE  CONTROL  timer  works  only  on  problem  frames  (such  as  81, 

83,  93  In  Figure  6.)  It  would  not  work  on  the  "wrong  answer"  frame  such 
as  82,  so  the  student  who  is  working  with  a low  setting,  such  as  3,5  secs, 
gefs  a "breather"  If  he  makes  an  error,  at  least. 

The  student  who  gets  all  7 problems  correct  emerges  at  frame  95, 
where  he  Is  congratulated  by  the  audio  (A-68)  and  offered  the  opportunity 
to  begin  another  similar  sequence,  perhaps  at  a lower  time  setting.  In 
one  AVIT  program  there  are  three  such  seven-question  sequences  in  a row, 
and  if  the  student  wants  more  practice  after  completing  all  three,  he  can 
elect  to  go  back  to  the  first  set  of  exercises,  and  so  on. 

Some  appreciation  of  the  flexibility  of  the  audio  system  used  in  the 
AVIT  may  be  obtained  from  the  last  two  patterns  discussed,  particularly 
that  shown  In  Figure  6.  Visual  stimulus  presentation  devices  that  permit 
flexible  presentation  have  been  available  for  some  time,  but  audio  and 
audio-visual  devices  have  generally  been  locked  into  a fixed  sequence.  In 
contrast  consider  how  the  audio  apparatus  in  the  AVIT  must  function  to 
permit  the  pattern  shown  in  Figure  6.  When  frame  81  appears  on  the  screen, 
the  audio  device  (a  tape  player)  has  the  tape  so  positioned  as  to  be  ready 
to  play  audio  message  A-61.  If  the  student  gives  the  right  answer,  the 
AVIT  logic  switches  the  tape  player  to  FAST  FORWARD  and  runs  to  the  be- 
ginning of  audio  message  A-62,  but  does  not  play  it.  If  an  error  is  made 
(on  frame  8!)  the  tape  player  simply  switches  to  PLAY  and  plays  A-61, 
stopping  automatically  at  the  end  of  the  message.  When  the  student,  on 
frame  82,  presses  the  trigger  swilch  to  return  to  frame  82,  the  fape 
player  must  switch  to  REWIND,  rewind  message  A-61  and  be  ready  to  play  it 
again.  If  the  student  makes  an  error  on  frame  93  and  so  goes  to  frame 
94,  the  tape  recorder  will  play  audio  message  A-67,  of  course.  When  the 
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student  presses  the  trigger  to  return  to  frame  81,  however,  the  tape 
player  must  now  rewind  6 messages,  and  stop  In  a position  ready  to  play 
A-61  again, 

DIDACTIC  MODE 

Mention  was  made  of  ■‘he  five  lettered  buttons  A,  B,  C,  D,  and  R. 

The  AVIT  Is  a further  development  or  modification  of  the  AutoTutor  Mark  4, 
which  Is  a branching  teaching  machine.  The  student  using  a Mark  4 
responds  to  multiple  choice  questions  by  pressing  buttons  and  the  Mark  4 
"branches"  the  student  to  appropriate  visual  material  on  the  basis  of  the 
student’s  response.  A similar  capability  Is  built  Into  the  AVIT,  and 
allows  it  to  operate  In  the  DIDACTIC  mode.  The  DIDACTIC  mode  has  two  major 
purposes: 

1.  First,  *or  instructional  sequences  where  a multiple  choice  question 
Is  appropriate.  For  example,  the  visual  presentation  might  be  the  altimeter 
alone,  and  the  student  would  be  required  to  select  the  altitude  shown  from 

4 choices  appearing  as  numbers  on  the  screen,  witn  a button  designation 
beside  each.  The  student  choosing  the  right  answer  would  advance  to  the 
next  instruction  frame  and  question.  For  the  student  who  makes  an  error, 
two  options  are  available.  In  the  DIDACTIC  mode,  the  AVIT  can  operate  in 
a full  branching  sub-mode,  so  that  each  button  leads  to  a different  frame, 
or  in  a "right-wrong"  branching  mode  In  which  the  right  answer  leads  to 
one  particular  frame,  all  wrong  answers  to  a single  second  frame.  The 
choice  depends  on  whether  a single  correctional  frame  Is  appropriate  for 
all  the  errors,  or  whether  It  is  desired  to  supply  specific  correctional 
material  for  each  error.  [The  reader  will  recognize  that  the  sequences 
shown  for  the  TRAINER  mode  all  use  "right-wrong’’  branching.  In  fact, 
the  full  branching  option  is  available  in  the  TRAINER  mode  also,  but  we 
found  no  practical  use  for  It  In  the  TRAINER  mode  and  would  not  propose  to 
Include  this  option  in  an  AVIT  designed  for  helicopter  instrument  training.! 
In  any  event,  the  inclusion  of  the  OIDACTfC  mode  of  operation  allows 
"conventional"  branching  programmed  instruction  to  be  freely  intermixed 
with  instruction  in  the  TRAINER  mode. 

2,  The  second  use  made  of  the  DIDACTIC  mode  Is  where  the  student  Is 
offered  options  as  to  how  he  wishes  to  proceed  through  the  program.  Such 
choice  points  were  shown  In  the  program  segments  shown  in  Figures  5 and 
6,  where  the  student  may  elect  frame  54  to  go  through  an  instructional 
sequence  he  is  "entitled"  to  skip  by  having  passad  the  pre-test  for  the 
maneuver,  and  in  the  segment  shown  in  Figure  6,  frame  95,  where  the 
student  may  elect  to  take  another  test  sequence  or  to  go  on  to  the  next 
maneuver. 
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OTHER  MOOES 


Strictly  speaking,  the  AVIT  has  several  other  modes  of  operation  which 
will  be  mentioned  briefly.  They  and  their  uses  are  largely  self-explanatory. 

MANDATORY  FWD  mode  (MF)  no  option,  used  for  procedural  frames. 

Excursion  is  initiated  by  pressing  the  trigger  switch.  Frames 
40,  43,  47,  49,  51  and  53  in  Figure  5 are  examples. 

MANDATORY  RVS  mode  (MR)  No  option,  usod  for  procedural  frames. 

Excursion  is  initiated  by  pressing  the  trigger  switch  or 
the  R button.  Frames  82,  84,  86,  88,  90,  92  and  94  in 
Figure  6 are  examples. 

AUTO  ADVANCE  mode  (AA)  In  this  mode,  the  AVIT  advances  automatically 
to  the  next  frame  (in  the  logical,  not  necessarily  the  numerical 
^sequence)  as  soon  as  the  sound  message  for  the  frame  is 
finished.  This  mode  has  two  uses.  First,  it  allows  an  AVIT 
program  to  contain  segments  of  conventional  audio-visual 
material  similar  to  that  used  with  synchronized  sound-slide 
devices,  and  this  frequently  a useful  programming  resource. 

The  second  use  of  the  AUTOMATIC  ADVANCE  mode  is  for  excursions 
that  are  too  long  to  be  covered  in  a single  jump.  The  longest 
single  excursion  possible  in  the  AVIT  for  the  visual  material 
Is  bT”frames  forward  or  backward,  and  31  messages  forward  or 
29  backwards  on  the  audio.  If  a longer  excursion  is  required, 
the  programmer  simply  includes  an  AUTOMATIC  ADVANCE  frame  at 
the  appropriate  point,  and  automatically  begins  a new 
excursion  from  that  point.  With  this  resource  excursions 
of  any  reasonable  length  become  possible. 

It  should  be  re-emphasized  that  the  selection  cf  the  desired  mode  on 
a particular  frame  is  entirely  automatic,  requiring  no  attention  from  the 
student  or  from  a supervising  instructor.  The  only  control  the  student  or 
instructor  has  occasion  to  use  on  the  AVIT  (except  for  the  response  devices, 
of  course)  is  the  EXPOSURE  CONTROL  switch. 
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APPENDIX  H 

A Recommended  Deslon  for 
Experimental  Studios  Us* no  ASUPT 


R,  P.  Oemarne 

Brief,  Described  herein  is  a design  which  lends  Itself  to  studies  of 
the  relationships  of  piloting  performance  fo  conditions  of  simulation  provided 
by  ASUPT,  The  information  yielded  by  the  design  and  the  way  in  which  it  works 
are  described  for  an  Illustrative  study,  Apnl icatlons'and  extensions  of  the 
model,  though  they  remain  to  be  specified,  are  touched  on  briefly. 


General . In  the  training  of  pilots,  as  well  as  in  nerformance  studies 
us i ng  experienced  pilots,  the  conditions  under  which  piloting  tasks  are 
performed  in  ASUPT  wi U often  be  of  an  incremental  nature.  In  performance 
studies  involving  highly  proficient  pilots,  however,  progressive  degradations 
or  reductions  in  features  of  simulation  may  be  introduced  for  the  purpose  of 
determining  the  extent  to  which  performance  of  piloting  tasks  depends  on  such 
conditions.  In  either  of  these  cases,  the  design  described  in  this  paper 
would  apply. 

Characteristics  of  the  desion.  The  dosion  calls  for  stepwise  increments 
or  decrements  in  simulation  with  an  equal  nunbor  of  observations  of  nerformance 
at  each  step.  Each  step  will  be  spoken  of  as  a condition  of  simulation. 

Each  condition  is  either  present  or  absent,  buf  a sten-down  or  sfen-un 
arrangement  holds,  whereby  the  presence  of  a particular  condition  implies 
that  conditions  up  to  that  point  are  present,  or  conversely,  that  conditions 
down  to  that  point  are  absent. 

With  the  above  arrangement  of  sets  of  conditions  under  which  pilottnq 
performance  is  observed,  an  estimate  of  the  multiple  linear  regression  co- 
efficients for  oredicinq  a pilot’s  performance  score  on  a given  maneuver 
under  a gi'en  set  of  conditions  can  he  obtained  very  easily  from  the 
difference  of  his  scores  under  adjacent  sots  of  conditions,  Such  estimates 
can  be  extended  to  differences  among  corresponding  scores  when  more  than  one 
performance  score  is  obtained  for  a given  maneuver  under  different  sets  of 
conditions  of  simulation. 

When  two  or  more  observations  of  performance  are  obtained  for  a single 
pilot  under  each  set  of  conditions,  the  estimated  multiple  linear  regression 
coefficients  are  given  by  the  mean  of  the  difference  scores  under  adjacent 
sets  of  conditions.  This  also  applies  to  studies  in  which  performances  are 
observed  under  each  set  of  conditions  for  two  pr  more  pilots.  The  mean 


performance  Is  computed  for  each  set  of  conditions,  Each  estimated 
regression-coefficient  Is  then  equal  to  the  difference  In  mean  performance 
for  a pair  of  adjacent  sets  of  conditions.  It  might  be  noted  here,  even 
though  the  example  given  later  will  make  this  much  clearer,  that  the  step- 
up  (or  step-down  , If  one  wishes)  arrangement  of  conditions  Implies  that  If 
there  are  p conditions  there  will  be  p+l  sets  of  conditions.  Further,  since 
the  regression  coefficients  are  based  on  the  differences  between  two  scores 
the  same  results  are  obtained  for  raw  scores  as  for  scores  expressed  In 
mean-deviation  form.  Specifically,  ( Yj  - Y2)  - C(Y|  - Y)  - (Y2  - Y)]. 

An  example  of  how  the  design  works.  In  the  example  which  follows, 
there  Is  a s+ep-up  arrangement  for  four  conditions  of  simulation.  The  five 
sets  of  conditions  under  which  performance  of  a given  piloting  task  Is 
observed  on  a single  pilot  can  be  represented  by  a matrix,  as  follows: 
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bj  Is  the  4x1  veclor  of  estimated  multipie  linear 
regression  coefficients,  based  on  Trial  I. 


Is  the  performance 
score  under  the  l'b 
set  of  conditions 
for  Trial  I by  a 
single  pi  lot, 

(I  = 1,2. 3, 4, 5), 
Is  slml larly  de- 
flnd  for  Trial  2, 

Is  the  5x4  matrix 
which  denotes  the 
conditions  under 
which  performances 
are  observed. 


^2  Is  similarly  defined  for  Trial  2 
* 

b Is  the  4x1  vector  of  estimated  regression  coefficients, 
~~  based  on  Trials  I and  2, 
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where  Yf  Is  the  mean  of  the  performance  scores 'under  the 
+h  • 

I set  of  conditions. 
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The  least-squares  prediction  equation  takes  the.  form: 

A 

Y * x b , where 

- nn  - 

1,  = » X vector  of  p-adlcted  performance 


V-l 


L:LJ 


scores  In  mean-deviation  form; 


x Is  the  10x4  matrix  of  predictor  variables  In  mean-deviation  form, 
representing  the  sets  of  conditions  under  which  the  observed  performance 
scores  were  obtained;  and,  b Is  the  4x1  vector  of  regression  coefficients. 
The  ratio  of  the  variance  oT  the  predicted  performance  scores  to  the 
variance  of  the  observed  performance  scores  Is  equal  to  the  squared  co- 
efficient of  multiple  correlation. 


* i,  v’Y 
Y FTil 


and  N Is  the 


number  of  observations  (10  lr.  the  present  case). 


Due  to  the  fact  that  the  matrix  of  Independent  or  predictor  variables 
forms  what  Is  called  an  equally-spaced  simplex,  the  prediction  equation  Is 
simply 


YIJ  * Yl.  » 

where 


Y. . Is  the  predicted  performance  score  under  the  Ith  set  of 
J conditions  on  the  j+h  trial,  and 

Yj  Is  the  mean  performance  score  under  the  1^  set  of  conditions. 

From  the  above  It  can  perhaps  be  seen  that  If  a pilot’s  performance  under 
each  set  of  conditions  were  Identical  for  both  trials,  Yj#  would  bo  eaual 
to  Yjj  and,  as  a consequence,  Ry,i234  wou^  be  equal  to  1 loo. 


The  example,  described  above,  could  be  applied  just  as  readily  to  two 
pilots  for  whom  one  performance  score  was  obtained  under  each  scf  of 
conditions,,  as  In  the  present  case  of  one  pilot  ♦or  whom  two  performance 
scores  were  obtained  under  sach  seT  of  conditions.  The  extension  to  more  than 
four  conditions  of  simulation  Is  straightforward,  as  long  as  the  same  number 
of  observations  of  performance  Is  obtained  for  each  of  the  p+l  sets  of 
conditions,,  When  this  Is  not  the  case,  the  multiple  linear  prediction 
equation  Is  not  as  simple  as  In  the  example  given.  Specifically,  a predicted 


score  under  fhe  1th  set  of  conditions  would  then  be  based  on  observed 
performances  under  the  (1-1),  I,  and  (t+l)  sets  of  conditions,  except 
for  1=1,  In  which  case  only  1 and  (i+l)  apply  and  I=p+I,  In  which  case 
only  ( l-l ) and  I apply. 

Further  applications  of  the  simplex  design.  An  Interesting,  and 
perhaps  highly  useful,  application  of  the  simp  lei  design  would  be  one  In 
which  the  conditions  of  simulation  are  permitted  to  vary  so  that  equal 
Increments  In  performance  are  obtained.  The  regression  coefficients 
wou'd  then  be  equal  In  value  and  the  question  of  Interest  would  be  to 
ascertain  the  conditions  of  simulation  e.g.,  the  k values  which  range  from 
0 to  I for  components  of  morion  simulation,  required  for  attaining  equal 
Increments  (or  decrements)  In  observed  performances.  The  usual  model  for 
multiple  linear  regression  would  apply  directly,  except  that  the  Increments 
In  performance  would  bo  required  to  be  equal  and  the  conditions  of 
simulation  would  become  the  dependent  variables. 

Finally,  It  Is  worthy  of  note  that  the  simplex  design  as  described 
In  this  paper  is  a fully-crossed  design  Inasmuch  as  each  pilot's 
performance  Is  observed  under  all  sets  of  conditions,  trials,  and  tasks. 

The  resulting  data  for  a given  performance  score  (dependent  variable) 
lend  themselves  to  an  analysis  of  variance  which  leads  to  estimated 
components  of  variance.  Computer  programs  are  already  available2  for 
such  analyses  or  can  be  readily  developed. 


Cornelius,  E.  T.,  Woodward,  J.  A.,  and  Demaree,  R.  G«,  CRONB: 

A Fortran  IV  program  to  compute  variance  components  for  various 
experTmenlal  designs.  'Port  wVrfh,  Tex.*;  Insttlu^e  of  Behavlor'a I 
^es'earcfi7"  Texas'  "CnTTst  fan  University,  March,  1972. 
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